A novel segmentation algorithm for natural color image is proposed. Fibonacci Lattice-based Sampling is used to get the color labels of image so as to take advantage of the traditional approaches developed for gray-scale images. Using local fuzzy homogeneity derived from color labels, texture component is calculated to characterize spatial information. Color component is obtained by peer group filtering. To avoid over-segmentation of texture areas in a color image, these color and texture components are jointly employed to group the pixels into homogenous regions by the mean shift based clustering. Finally, experiments show very promising results.