In this work, Au/SiO2/NiO/SiO2/Si structure charge trapping memory using NiO as the charge trapping layer was fabricated, and the impacts of the annealing temperature on the charge trapping memory performance were investigated in detail. The sample thermal annealed at 750°C indicated a large memory window of 2.07 V under a low sweeping voltage of ±5 V, which also has excellent charge retention properties with only small charge loss of ∼4.9% after more than 104 s retention. The high resolved transmission electron microscopy shows that the NiO films grew as nano-pillars structure. It is proposed that the excellent memory characteristics of the device are attributed to the inherent atomic defects and oxygen vacancies accumulated by the grain boundaries around NiO nano-pillars. Meanwhile the interface inter-diffusion formed by thermal annealing process is also an indispensable factor for the excellent memory characteristics of the device.