In this paper we present an automatic hand gesture recognition system operating on video stream. The system consists of two modules: hand gesture detection module and hand gesture recognition module. The detection module could accurately locate the hand regions with a blue rectangle; this is mainly based on Viola-Jones method, which is currently considered the fastest and most accurate learning-based method for object detection. In the recognition module, the Hu invariant moments feature vectors of the detected hand gesture are extracted and a support vector machines (SVMs) classifier is trained for final recognition, due to its high generalization performance without the need to add a priori knowledge. The performance of the proposed system is tested through a series of experiments and a simple human-computer interaction application based on hand gesture recognition method is finally developed.