In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel, facilitated with Kalman filtering, to continuously update the position and the orientation of the moving vehicles. To further reliably track vehicles under partial occlusion or even total occlusion, our tracking algorithm also systematically builds 3-D vehicle model, from which the license plate region is identified and a self-similarity descriptor is further used for low-resolution license plate matching. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles under serious occlusion while maintaining the knowledge of 3-D geometry of the tracked vehicles.