Previously, we proposed an auto-regressive hidden Markov model (AR-HMM) and an accompanying parameter estimation method. An AR-HMM was obtained by combining an AR process with an HMM introduced as a non-stationary excitation model. We demonstrated that the AR-HMM can accurately estimate the characteristics of both articulatory systems and excitation signals from high-pitched speech. As the parameter estimation method iteratively executes learning processes of HMM parameters, the proposed method was calculation-intensive. Here, we propose two novel kinds of auto-regressive, non-stationary excited signal parameter estimation methods to reduce the amount of calculation required