In this paper a fast and robust visual tracking approach based on GPU acceleration is proposed. It is an effective combination of two GPU-accelerated algorithms. One is a GPU accelerated visual tracking algorithm based on the Efficient Second-order Minimization (GPU-ESM) algorithm. The other is a GPU based Scale Invariant Feature Transform (SIFT) algorithm, which is used in those extreme cases for GPU-ESM tracking algorithm, i.e. large image differences, occlusions etc. System performances have been greatly improved by our combination approach. We have extended the tracking region from a planar region to a 3D region. Translation details of both GPU algorithms and their combination strategy are described. System performances are evaluated with experimental data. Optimization techniques are presented as a reference for GPU application developers.