The shortcomings of inertial sensor-based localization for mobile robots are discussed and a method to improve the performance of localization is introduced. Because dead reckoning using wheel speed sensors is insufficient for accurate motion estimation in uneven and slippery environments, many studies have used inertial sensors in order to improve the performance of localization. However, inertial sensor-based localization also has inherent problems caused by an accumulation of sensor errors. To improve the performance of inertial sensor-based localization for mobile robots, a method is suggested that involves the reduction of unexpected noise of inertial sensor using a Multiple Estimation Windows (MEW) filter. The main concept of the MEW filter is to estimate the long-term signal of interest using a single long-term moving window and multiple short-term moving windows. The proposed denoising method can reduce the noise of all frequencies without the loss of information from the signal of interest. The performance improvement of the mobile robot's localization using the MEW filter is confirmed experimentally.