In this article, we propose a new particle filtering scheme, called a switching particle filter, which allows robust and accurate visual tracking under typical circumstances of real-time visual tracking. This scheme switches two complementary sampling algorithms, Condensation and Auxiliary Particle Filter, in an on-line fashion based on the confidence of the filtered state of the visual target. The accuracy and robustness of the switching scheme were evaluated using real visual tracking experiments as well as computer simulations. Furthermore, we demonstrate through visual tracking experiments that our scheme not only outperforms existing particle filters but also assists on-line learning of target dynamics.