The handwritten character recognition (HCR) is the major problem in the character recognition domain. There are a lot of methods applied to the handwritten character recognition problems. The "Extreme Learning Machine" (ELM) is the one among them. ELM is the single hidden layer neural networks widely applied in many applications and classification problems. The features of ELM are faster learning method, and it has better performances when compared with other gradient-based neural networks algorithms. In previous research studies, they applied ELM in the field of image processing such as face recognition and face detection. In addition, ELM was applied in many character recognition research studies and it has a good performance. In this paper, this paper used the modified version of generalized radial basis function ELM (MELM-GRBF) to recognize the handwritten characters. Moreover, this paper proposes the improving version of MELM-GRBF for HCR by using the semi-optimization scheme to select the better centers for RBF kernel. The experiments in this paper were applied in three handwritten datasets including Thai characters, Bangla numerals and Devanagari numerals. In the experiment results, the propose method has generally better performances when compared with ELM, MELM-GRBF.