A new method of optical encrypted color images based on a spectral fusion (segmented) of information has been proposed. Color images are decomposed into three basic color components. Pertinent information fusion is considered as segmentation. Encrypting consists in modulating each of these three components of the image spectrum by a corresponding phase mask. This phase mask includes pertinent information collected from various encrypted color keys according to a fusion criterion. As encrypting keys are formed by real images, it becomes an impossible task to get the target image by any trial and error random images techniques. For decrypting, a simple algorithm based on composite correlation filters is proposed. Simulation results corroborate the effectiveness of our algorithm.