In this paper, a new algorithm to estimate confidence measure of support vector machine (SVM) is presented. The algorithm computes the distance from testing sample to the optimal hyperplane of SVM, and the probability that the testing sample and its k nearest neighbors belong to the same class as the decision of Libsvm for the testing sample. The algorithm rejects the classification results of samples whose confidence measures are smaller than the threshold corresponding to a given rejection rate. Experiments show that the performance of the SVM classifier has been improved using this algorithm.