In this paper, we demonstrate the need for dimensionality reduction to mitigate model overfitting on the nontrivial problem of gender classification from digital images. In (his study we explore four feature selection schemes using Genetic Algorithm, Memetic Algorithms, and Random Forest, which are fed to a nonlinear support vector machine (SVM) for final classification. The performance of the model (feature) selection approaches are evaluated against two distinct datasets of facial images: FG-NET which contains toddlers to seniors and the UIUC-PAL which contains faces of adults up to seniors. This work demonstrates that feature selection can, and does, improve performance of an SVM based gender classification system significantly.