Facing the problem of poor detection effect and bad real-time performance of existing method, a fast and robust face detecting and tracking algorithm is proposed, which detects face region by a improved Adaboost method at first, and then tracks it by Mean Shift algorithm combined with the motion history image (MHI). The experimental results demonstrate that the proposed algorithm can robustly detect and track human face, even in the event of occlusion, hence suitable for real-time video surveillance.