It is key technique to improve the recognition efficiency while not affecting the recognition accuracy in the speech recognition systems. The method combining Viterbi algorithm with Beam pruning technique is useful to compress the search space, which reduces the computational complexity. However, Viterbi-Beam algorithm is a kind of suboptimal algorithm. The selecting of the pruning threshold will affect the system recognition accuracy. In this paper, we propose an adaptive Viterbi-Beam search algorithm by analyzing the voice activity model of different stages. This method leads to an improvement on search efficiency of 35.77%, without the recognition accuracy reduction.