Large appearance changes in visual tracking affect the tracking performance severely. To address this challenge, in this paper we develop an effective appearance model with the highly discriminative features. We propose an online Fisher discrimination boosting feature selection mechanism, which selects features that reduce the with-in scatter while enlarging the between-class scatter, thereby enhancing the discriminative capability between the target and background. Moreover, we utilize a particle filtering framework for visual tracking, in which the weights of candidate particles take into account the context information around the particles, thereby enhancing the robustness of tracking. In order to increase efficiency, a coarse-to-fine search strategy is exploited to efficiently and accurately locate the target. Extensive experiments on the CVPR2013 tracking benchmark demonstrate the competitive performance of our algorithm over other representative algorithms in terms of accuracy and robustness.