We report significant difference in the Raman spectra of two different kinds of CaB 6 single crystals grown from boron purity 99.9% (3N) or 99.9999% (6N), respectively. Our Raman spectra of CaB 6 (3N), which are similar to those of previous measurement [N. Ogita, S. Nagai, N. Okamoto, M. Udagawa, F. Iga, M. Sera, J. Akimitsu, S. Kunii, Phys. Rev. B 68 (2003) 224305], show peaks at 781.3cm −1 (T 2g ), 1140.1cm −1 (E g ), and 1283.5cm −1 (A 1g ). The E g mode shows a characteristic double-peak feature due to an additional weak broad peak centered at 1156.0cm −1 . However, the Raman spectra of CaB 6 (6N) show sharp peaks at 772.5cm −1 (T 2g ), 1137.9cm −1 (E g ), and 1266.6cm −1 (A 1g ). The peak frequencies are down shifted as much as 17cm −1 . In addition, no additional peak feature is observed for the E g mode so that the mode is symmetric in the case of CaB 6 (6N). The X-ray powder diffraction patterns for both CaB 6 (3N) and CaB 6 (6N) show that the lattice parameters are essentially the same. The majority of the impurity in the 99.9% (3N) boron is assessed to be C. Thus we prepared Ca(B 0.995 C 0.005 ) 6 , CaB 6 (6N) doped with C, and looked for the difference in the Raman spectra. The Raman spectra of Ca(B 0.995 C 0.005 ) 6 are nearly identical to those of CaB 6 (6N), indicating that the difference in the Raman spectra of CaB 6 (3N) and CaB 6 (6N) is not due to C impurity. However, presence of impurity, even if small amount, seems to be enough to trigger local-structure changes to lower symmetry inducing the difference in Raman spectra of CaB 6 (3N) and CaB 6 (6N).