In this paper, we propose a probabilistic approach to verifying the meaningful phrases, named concepts, in a user utterance. In this approach, a concept is verified not only according to its confidence measures assessed by various confidence measurement methods hut also according to neighboring concepts and their confidence levels. Two corpora are used to evaluate the proposed approach. One is collected from landline phone calls, and the other is collected from wireless phone calls. Experimental results show that the proposed model significantly outperforms the models using only confidence measures. The relative reduction in verification error is 23.7% on the corpus of landline phone calls, and 34.2% on the corpus of wireless phone calls. These substantial improvements show the proposed probabilistic verification approach is effective in integrating contextual confidence information and different confidence measures to verify the concepts of user utterances.