A new multi-class image recognition method based on Bayes point machines (BPM) and binary tree is proposed. Analysis and experimental results demonstrate that BPM is a good alternative to the popular support vector machine (SVM). Also we designed a novel multi-class method by utilizing both class distances and class distributions. The integrated classification procedure starts with computing all the one-to-rest distances and distributions, and then constructs the binary classifying tree for BPM classification. Experimental results on various standard image databases show that the proposed scheme has significant improvement of classification accuracy compared to traditional SVM-based schemes while requiring much fewer RVs and decision time.