The damage of road surface reduces its service life. In order to improve road maintenance and management efficiency, detection and recognition of pavement are studied based on video images in this paper. Firstly, we collect a large number of road surface images of 3 different conditions including transverse crack, longitudinal crack and turtle crack separately to construct road surface conditions library. Secondly, deal the road damaged image with gray, gray transform and image smoothing. Then, use mathematical morphology method to deal with crack image and projection to identify crack category. Finally, develop the pavement crack recognition software based on Matlab. Selecting the pavement samples for experiment, the results show that this identification algorithm can accurately identify the category of crack.