Collision detection is very important to enhance the sense of reality and immersion in virtual environment. Most of the traditional collision detection algorithms have been analyzed, but there is no algorithm that is applicable to all situations, and with the scene complexity increases, the efficiency of the algorithm tends to decline rapidly. In this paper, a new method is proposed to solve the problems: converting the problem of collision detection to the nonlinear programming problem with constraint conditions, and then using the adaptive genetic algorithm to solve it. The experiment results show that this method is efficient, especially in large-scale scenes.