Time-varying frequency characteristic is extracted from the average Mel cepstrum, and the cepstrum value series on the frequency are obtained. The deterministic component and stochastic component of the time series are separated from the series. As zero mean autocovariance nonstationary time series, the stochastic component is analyzed by full order TVPAR (Time-Varying Parameter Autoregressive) model, and the characteristic parameters are extracted from speech signals of a speaker. Then the speech signals are recognized on the stochastic component of the time series and after nonstationary time series analysis by full order TVPAR model. The experimental results manifest that the recognition rate obtained by full order TVPAR model are higher than only on stochastic component of the time series, with one or two characteristic frequencies the average recognition rates reach 94.13% and 99.6% respectively.