The goal of visual tracking is to estimate the location of the visual target in a sequence of images. If the video has long-term sequence of images, the tracking task without prior information is severely difficult due to the occlusion or the disappearance. Recently, tracking-by-detection methods have been proposed to solve the long-term tracking problem. However, most of them also suffer from drifting, since they did not sufficiently consider the labeling problem when updating the detector. In this paper, a robust tracking approach via a novel partial update strategy is proposed. The proposed strategy is based on the thinking that unoccluded area of the target still includes useful information, while the use of the whole area will degrade the performance of the detector. As a result, the proposed partial update strategy effectively improves the performance of the tracking-by-detection method in the presence of partial occlusions and abrupt changes of illumination.