In order to improve EFVS (Enhanced Flight Vision System) in visible light image and infrared image fusion efficiency, to enhance the real-time of Wavelet transform image fusion algorithm, put forward a kind of wavelet transform image fusion algorithm based on matrix Doolittle decomposition. The pixel matrix of the algorithm on the original image is decomposed by Doolittle, Wavelet fusion of upper triangular matrix Doolittle decomposition of image matrix, then using Doolittle matrix inversion to obtain the fusion image. Experimental results show that the algorithm can get better fusion effect, while ensuring a better real-time.