There have been many proposals to extend the basic two-class SVM classifier for multiclass classification and it is established that among these extensions binary-structured hierarchical SVMs is the most efficient computationally. However, determining an effective binary structure by recursively dividing the classes is a major research issue. We describe a new classifier, GP-SVM, based on greedy partitioning of classes and demonstrate that GP-SVM gives better classification accuracy than all major combinational techniques besides having the computational advantages. The advantages of GP-SVM is better realized when the number of classes is large. We demonstrate this advantage in recognition of printed Odia character. We built a corpus of 10025 tagged Odia aksharas collected over multiple printed documents of different fonts. We used a very modest number of features. GP-SVM with 133 classes yielded 95% accuracy of recognition. During the learning process of GP-SVM, the proposed system could learn the taxonomy of character-shapes of Odia script.