Dim small targets detection in infrared image is one of the key techniques of infrared search and track system. In order to overcome the deficiencies of the detection algorithm based on gray-scale efficient targets detection method is proposed based on the directional discreteness. Firstly, targets are separately detected by two methods based on gray-scale characteristics and direction discreteness, and then merge the detection result to attain suspicious target set. Secondly, candidate targets are been accurate positioning according to the variance growth. Finally, the true targets are obtained according to the principle of moving continuity and trajectory consistency of moving targets in the image sequences. The experimental results indicate that the method can effectively detect the small and weak targets under complex backgrounds.