This paper proposes a new method for tracking with accurate scale estimation using motion vector errors. First we estimate an object displacement vector using the set of inlier motion vectors(VMF) filtered out by Median Flow tracker [2]. Then we calculate each motion vector error, i.e. the distance between each VMF and the estimated object displacement vector. 50 percent of VMF with large motion vector errors are excluded from VMF and the other vectors become candidate vectors. Next, the scale change ratios of the object are calculated using candidate vectors. We calculate the errors of scale change ratio using the median of scale change ratios, using the same method as we did for motion vector errors. Finally, if the median of ratio errors is lower than a threshold value, the median of scale change ratios is decided to the object scale change ratio. Otherwise the object scale is maintained to the next frame.