We present an unsupervised segmentation algorithm combining the mean shift procedure and deterministic annealing expectation maximization (DAEM) called MS-DAEM algorithm. We use the mean shift procedure to determine the number of components in a mixture model and to detect their modes of each mixture component. Next, we have adopted the Gaussian mixture model (GMM) to represent the probability distribution of color feature vectors. A DAEM formula is used to estimate the parameters of the GMM which represents the multi-colored objects statistically. The experimental results show that the mean shift part of the proposed MS-DAEM algorithm is efficient to determine the number of components and initial modes of each component in mixture models. And also it shows that the DAEM part provides a global optimal solution for the parameter estimation in a mixture model and the natural color images are segmented efficiently by using the GMM with components estimated by MS-DAEM algorithm.