We propose a new tracking method based on a group sparsity learning model. Previous work on sparsity tracking rely on a single sparse model to characterize the templates of tracking targets, which is hard to express complex tracking scenes. In this work, we utilize a superposition of multiple simpler sparse models to capture the structural information across templates. More specifically, our tracking method is formulated within particle filter framework and the particle representations are decomposed into two sparsity norms: a $$l_{1,\infty }$$ l 1 , ∞ norm and a $$l_{1,2}$$ l 1 , 2 norm, capturing the common and different information across the templates, respectively. To efficiently implement the proposed tracker, we adapt the alternating direction method of multipliers to solve the formulated two-norm optimization problem. The proposed tracking method is compared with seven state-of-the-art trackers using 16 publicly available and challenging video sequences due to appearance changes, heavy occlusions, and pose variations. Experiment results show that our tracker outperforms the five other tracking methods.