3D upper body tracking and modeling is a topic greatly studied by the computer vision society because it is useful in a great number of applications such as human machine interface, companion robots animation or human activity analysis. However there is a challenging problem: the complexity of usual tracking algorithms, that exponentially increases with the dimension of the state vector, becomes too difficult to handle. To tackle this problem, we propose an approach that combines several annealing particle filters defined independently for each limb and belief propagation method to add geometrical constraints between individual filters. Experimental results on a real human gestures sequence will show that this combined approach leads to reliable results.