In this paper we propose a new solution for representing and tracking crowded traffic environments by using dense stereo data. The proposed method relies on the information provided by two compact 2.5D grid-based representations: a classified occupancy grid and an intensity grid. The measurement data is extracted using a predefined Policy Tree, which represents a path structure used to accelerate the object delimiter extraction. The extracted measurements are given in form of rectangular grid blocks that are described by three components: a dynamic, a geometry and an intensity component. We propose a medium level tracking approach in which the state is estimated for each block individually. To be able to work with a high dimensional state space a Rao-Blackwellized particle filter is used. The proposed solution has several advantages. First, the data association is performed at the particle level, thus being handled in a natural way by a weighting-resampling mechanism. Second, unlike other existing geometry-based solutions we also incorporate the intensity information in the tracking process. Finally, the proposed method takes into account the uncertainties of the stereovision system.