Object semantic reduces the semantic gap in Content Based Image Retrieval (CBIR). In recent years, numerous methods for object semantic categorization have been proposed. Semantic segmentation is a key factor affecting the accuracy of object semantic categorization. The existing semantic segmentation methods usually chose pixel or super-pixel as the processing input. But the information contained in a pixel or a super-pixel is usually not enough to obtain the output including object semantic. Considering this problem, this work proposes a novel model to categorize object semantic in a color image. Firstly, we segment the region of interest (ROI) based on K-means, and extract the low-level features for the ROI. We use three kinds of features to train a support vector machine (SVM) classifier, and propose a re-weighting algorithm to integrate three SVM classifiers for categorizing the object semantic. The experiments on the publicly available image dataset MSRC21 and Corel show that the proposed method is a competitive method of object semantic categorization.