This paper presents a novel approach of hybridizing two conventional machine learning algorithms for dimension reduction. Genetic algorithm (GA) and support vector machines (SVMs) are integrated effectively based on a wrapper approach. Specifically, the GA component searches for the best attribute set using principles of evolutionary process, after which the reduced dataset is presented to the SVMs. Simulation results show that GA-SVM hybrid is able to produce good classification accuracy and a high level of consistency. In addition, improvements are made to the hybrid by using a correlation measure between attributes as a fitness measure to replace the weaker members in the population with newly formed chromosomes. This correlation measure injects greater diversity and increases the overall fitness of the population.