Texture images can be characterized with key features extracted from images. In this paper, the scale invariant feature transform (hereinafter SIFT) algorithm is utilized to generate local features for texture image classification. The local features are selected as inputs for texture classification framework. For each texture category, a texton dictionary is built based on the local features. To establish the texton dictionary, an adaptive mean shift clustering algorithm is run with all local features to generate key features (called textons) for texton dictionary. The texton dictionaries among texture categories are supposed be distinctive from each other to provide a highest performance in term of classification accuracy. A framework is proposed for classifying images into corresponding categories by matching their local features with textons from the texton dictionaries. This can be done by a histogram model of “match” vectors versus texture categories. Finally, our texture image database and the Ponce texture database are used to test the proposed approach. The results indicate a potential of our proposed method based on high classification accuracies achieved. They are 100% with our testing database for both classification and retrieval and 92 % and 100% with Ponce database for classification and retrieval, respectively.