This paper presents a high integrated and efficient embedded smart tracker with an Omni-vision tracking method to realize a real-time multi-object tracking. To achieve stable tracking, we improve a tracking method based on Mean Shift Embedded Particle Filter. An approach for image format conversion is used to provide front-end pretreatment. The tracker is composed of a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a CMOS image sensor and a Fisheye lens which can capture 180° view of the environment. Due to its small-size, the tracker could be flexibly applied to vehicle navigation, mobile monitoring and other related areas. We demonstrate the performance of the tracker and the method on several factors. Experimental results have been presented to show the accuracy and the robustness of the proposed system.