In this paper, a novel cryptographic system for color image security using chaotic Amplitude Phase Frequency Model (APFM) nonlinear adaptive filter is proposed. We set nine parameters, simulated time interval, and initial values for APFM nonlinear adaptive filter to generate chaos, and use the chaos property to design a color image encryption algorithm. The proposed scheme could encrypt the color image (plain image), and transform it into the color cipher image whose histograms of three components are almost uniformly distributed. The proposed scheme possesses a key sensitivity can reach the order of 10 to the power of −10 to get sufficient security strength to protect color image. Furthermore, our encryption algorithm can successfully pass the NIST SP 800-22a tests, which verifies the proposed scheme is a safe cryptographic system. The experimental results and security analyses demonstrate that the proposed method has a fine security performance.