With the pervasiveness of monitoring cameras installed in public places, schools, hospitals and homes, video analytics technologies for interpreting the generated video content are becoming more and more relevant to people's lives. Along this context, we develop a human-centric video surveillance system that identifies and tracks people in a given scene. In this paper, a parallel processing pipeline is proposed that integrates image processing modules in the system, such as face detection, person recognition and tracking, efficiently and smoothly, so that multiple people can be simultaneously tracked in real time. Furthermore, significant innovations are involved in this work in making each of the major image analysis modules both fast and robust to variations in pose, illumination, occlusions and so on. A demonstration software has been implemented that supports finding, tagging, identifying and tracking people in live or recorded videos with uncontrolled capturing conditions.