In order to rapidly and accurately determine the optimal threshold in video segmentation, an improved adaptive video motion segmentation method based on genetic algorithm is proposed. Through consummating the evolve mechanism, introducing adaptive initially individual, selection operator and adaptive judgments of termination, it can overcome the problems of poor astringency and premature occurrence in genetic algorithm. The experiment results show that the new segmentation method can greatly improve the speed of segmentation and can acquire good segmentation results.