Speaker identification based on Gaussian mixture model (GMM) has achieved good performance in practical applications. However its processing speed is very slow, which greatly affects its practicability. In this paper, we propose two methods to tackle this problem. Firstly, the GMM likelihood score formula is converted into linear, and further converted to the dot product of the model. Based on these conversions, we define the speaker metric space, and three fast speaker identification algorithms are proposed on the speaker metric space. Secondly, we propose a speaker retrieval method based on the high dimensional indexing technology, to further improve the speed of the speaker identification system. Experiments on the NIST2008 dataset show that the speaker identification method based on the speaker metric space can not only greatly improve the recognition speed, but also improve the performance of the speaker identification system. And the proposed speaker retrieval algorithm does facilitate the speed further.