The non-uniform illumination and irregularities in the surface of asphalt pavement make the conventional image enhancement and segmentation algorithm fail. In order to address these problems, we present an illumination invariant image enhancement and segmentation algorithm, which are crucial for feature extraction and classification of pavement crack image. The experimental results based on measured image by CCD camera on vehicle show that the presented algorithm is very effective for the illumination invariant and the irregularities in the surface of asphalt pavement.