Based on the analysis of all kinds of circumstances about vehicles crossing the center yellow line of the road, and the existing algorithm, a local comparison algorithm based on histogram was put forward. Namely, the yellow line region is firstly decided based on the sample image. Then it is used as a template to mask the sample image and the captured images. Further, the similarity of histogram is calculated on the template region on which whether the vehicle roll the yellow line is judged. Since the algorithm is only compared to the yellow line region, it is greatly reduced to the amount of calculation. And the influence on the judgment caused by image changes outside yellow area is avoided, by which the speed of recognition and accuracy is greatly improved. So the algorithm is simple, feasible.