The Tied Mixture Hidden Markov Model (TMHMM) is an important approach to reduce the number of free parameters in speech recognition. However, this model suffers from degradation in recognition accuracy due to its Gaussian Probability Density Function (GPDF) clustering error. This paper proposes a clustering algorithm called a Heterogeneous Centroid Neural Network (HCNN) for use in TMHMMs. The algorithm utilizes a Centroid Neural Network (CNN) to cluster acoustic feature vectors in the TMHMM. The HCNN uses a heterogeneous distance measure to allocate more code vectors in the heterogeneous areas where probability densities of different states overlap each other. When applied to an isolated Korean digit word recognition problem, the HCNN reduces the error rate by 9.39% over CNN clustering, and 14.63% over the traditional K-means clustering.