This paper presents a new minimum classification error (MCE)–mean square error (MSE) hybrid cost function to enhance the classification ability and speed up the learning process of radial basis function (RBF)-based classifier. Contributed by the MCE function, the proposed cost function enables the RBF-based classifier to achieve an excellent classification performance compared with the conventional MSE function. In addition, certain learning difficulties experienced by the MCE algorithm can be solved in an efficient and simple way. The presented results show that the proposed method exhibits a substantially higher convergence rate compared with the MCE function.