This paper describes about real-time object tracking method based on mean shift tracking algorithm. The mean shift tracking algorithm is an efficient technique for tracking object through. But the original mean shift tracking algorithm has a fixed spatial template. Therefore the tracking result is affected by changes of the tracked object. This paper presents an adaptive template modification strategy which overcomes the influence. The proposed method adjusts adaptively template by the Gabor wavelet. The tracker has the ability to handle the bandwidth matrix and the target angle. Experiments with several real worlds' sequences indicate our new method's capability to adapt to any combinations of the target's rotation, zoom and translation. With better description of the object it achieves much better precision.