Natural image classification is an important task. SIFT descriptors and bag-of-visterms (BOV) method have achieved very good results based on local image representation. Many studies use the support vector machine to classify and identify the image category after finished representation of the image. However, due to support vector machine (SVM) its own characteristics, it shows inflexible and less slow convergence rate. The selection of parameters influenced the results for the algorithm seriously. Therefore, this paper try to improve the image recognition performance by support vector machine algorithm based on ant colony algorithm. The method adopt dense SIFT descriptors and BOV method to obtain the image representation. In recognition step, we use the support vector machine as a classifier but ant colony optimization method is used to selects kernel function parameter and soft margin constant C penalty parameter. Experiment results show that this solution determined the parameter automatically without trial and error and improved performance on natural image classification tasks.