The text-independent speaker recognition system is mainly constituted of three functional modules including speech pretreatment, the feature parameter extraction, and pattern matching judgment. The paper uses the MATLAB software to acquire a design of the system. The feature parameters utilized in this paper are Mel-Frequency Cepstrum Coefficients (MFCC) and their first-order differential characteristics. With the help of the Fisher criterion the number of the dimension of the feature parameters is decreased. Vector Quantization (VQ) model is applied to devise the optimal codebook. The paper suggests some modifications in order to improve the efficiency of the algorithm on the basis of high recognition rate: Fisher ratio of each dimensional parameter is used as weighing coefficient at the distance measurement; an approach to speeding up the search is proposed; Process the empty cell in the procedure of codebook formation. In addition, it discusses a few factors including the training and testing time, the dimension of the codebook, stopping and acceptance thresholds, which have an impact on identification accuracy rate by experimentation.