Automatic gender recognition is an emerging problem in computer visions. An accurate gender recognition system can be used to reduce the search space in face recognition system for about half. However, since there is no definitive features of sexual dimorphism on human face that can be applied to all kind of face shapes from any race and age, it needs more studies to optimize the recognition system. Thus, this research investigates the implementation of complex-valued neural network as a classifier to recognize human gender which is based on face image. The experiment is also aimed to study the comparison between complex-valued and real-valued neural network. The methods proposed in this paper include image processing, feature extraction, and classification. After the face image processed by using local binary pattern to accentuate face texture and gradient filter to define face outline, the features of the face image is extracted by using histogram of oriented gradient. Then, the dimension of the resulted vectors is reduced by using principal component analysis. The final feature vectors is then used in neural network training and neural network testing processes. This paper shows investigation results in implementing the methods for some measurement parameters. The accuracy level of real-valued neural network system is a bit lower than the average accuracy level of complex-valued one, with 78.2% and 80.2% average accuracy rate, respectively. The results also show that complex-valued neural network system can achieve convergency rate four times faster than the real-valued neural network, which implies on the training process time.