A real-time vision-based object tracking system embedded in a quadrotor is presented in this paper. The integrated system is expected to enable the quadrotor to track a target object and keep with a fixed distance. A robust object tracking method via improving Kernelized Correlation Filters algorithm is proposed to achieve accurate visual tracking. In addition, a PID controller is applied to utilize the data from a RGB-D Camera. And the effectiveness of our proposed method is verified by experimental results.