Multi-sensor image fusion has attracted much attention in the remote sensing area. It urgently needs a universal and effective objective evaluation approach to measure the effect of image fusion. A new approach based on the singular value decomposition (SVD) is proposed for the remote sensing image fusion assessment. This method measures the divergence of the singular value features between the source images and fused image, and calculates energy distortion of the fused image from input images. By that means, the effect of the fusion algorithm is measured. Experiments are conducted from three aspects to confirm the idea. Firstly, when the source images include a SAR image, this method is more effective than Piella's evaluation methods and Xydeas's evaluation methods. Secondly, the experiments of different kinds of sensors and pixel-level fusion algorithms show that this objective evaluation appears highly consistent with the subjective evaluation. Lastly, the simple feature-level fusion images are measured by this objective evaluation method, and the results show coherence to the subjective factors. These experiments demonstrate its general effectiveness.