Sparse representation using the over-complete dictionary makes that decomposition coefficients are more sparse, and can reflect the inherent characteristics and structure of signals. A novel fusion method based on IHS transform and sparse representation for multi-spectral image and panchromatic image is proposed in this paper. Firstly, the IHS transform is applied to multi-spectral image. Then, the panchromatic image and the intensity component of multi-spectral image are fused through sparse representation. Sparse representation is used in local image patch instead of the whole image, because the size of image is usually large which is not suitable to sparse representation. Experimental results indicate that the fused image of the proposed method has high spatial resolution and spectral fidelity. And the proposed method is superior to the traditional methods based on the wavelet transform and wavelet frame transform.