In some researches of infrared(IR) and visible image fusion, the IR images often contribute more useful information. However, the IR sensor is sensitive to the temperature of a scene. Therefore, the IR images have low definition and contain much noise which affects the quality of the fused image. In a decomposed image Based on wavelet transform, the contrast of an image is proportional to the relative variation of the gray scale. And with the scale increasing, at least the mean and variance of impulse noise and Gaussian noise linearly decrease. Thus, a novel image fusion method Based on the wavelet transform is proposed in this paper. Firstly, both the IR image and visible image are decomposed by wavelet transform and their multi-scale sub images are achieved. Then, the contrast of IR image is improved by modifying the modulus of the sub images in scale space and stretching the dynamic scope of smooth sub image at coarser resolution level. Finally, the improved IR images and visible images are fused at different scales and reconstructed to the fused image. Experiments are carried out Based on discrete wavelet transform (DWT) and dual tree complex wavelet transform (DTCWT). The results turn out that the enhanced method is effective compared with the original methods.