This paper presents a way of improving the recognition rate of a typical Hidden Markov Model (HMM) -based Automatic Speech Recognition (ASR) system by integrating the l1 - least absolute deviation (LAD) algorithm and the l0 - least square (LS) algorithm in a framework designed to selectively use them based on the level of impulse noise present in speech signal. We present the overall architecture of the model, as well as experimental results and compare our enhanced noise-robust HMM-based ASR system with state-of-the-art proving the improvements brought by this approach as well as future directions of research.