It is a challenging task to develop a robust appearance model due to various factors such as partial occlusion, fast motion, background clutters and illumination variations. In this paper, we propose a novel target representation for visual tracking. Namely, a target candidate is represented by sparse affine combinations of dictionary templates in a particle filter framework. Affine combinations based target appearances can cover unknown appearances. In order to adapt the dynamic scenes across a video sequence, the dictionary templates are updated in the tracking process. Experimental results on several challenging video sequences against some state-of-the-art tracking algorithms demonstrate that the proposed algorithm is robust to illumination variations, background clutters, etc.