A novel evolutionary algorithm, which can be viewed as an extension to the simple, yet effective, approach of the Random-Mutation Hill Climber (RMHC), is presented. The algorithm addresses the shortcomings of RMHC and its multi-individual parallel version through the introduction of a penalty term into the fitness function, which penalizes individuals in the population for being too similar, hence maintaining population diversity. The performance of the algorithm is evaluated on the deceptive trap and a set of SAT problems, comparing them to the Crowding EA. The results show that at a small cost of solution speed on simpler problems, the algorithm gains better capabilities of dealing with the issues of local maxima.