A series of images of the whole wheel surface was captured by the imaging system. Median filtering and image contrast enhancement were used on the defect images, and then canny edge detection based on gradient histogram was done in further details. The images after edge detection are used for eight-neighbor boundary tracking, to obtain the sizes in-pixel of defects. The experiment system is set to test proposed method, and the actual sizes of defects are calculated. The results show that the detecting system can identify the wheel surface defects effectively, and obtain the defects' actual width and height accurately through image processing algorithm.