Soft-decision adaptive interpolation (SAI) provides a powerful framework for image interpolation. The robustness of SAI can be further improved by using weighted least-squares estimation, instead of least-squares estimation in both of the parameter estimation and data estimation steps. To address the mismatch issue of “geometric duality” during parameter estimation, the residuals (prediction errors) are weighted according to the geometric similarity between the pixel of interest and the residuals. The robustness of data estimation can be improved by modeling the weights of residuals with the well-known bilateral filter. Experimental results show that there is a 0.25-dB increase in peak signal-to-noise ratio (PSNR) for a sample set of natural images after the suggested improvements are incorporated into the original SAI. The proposed algorithm produces the highest quality in terms of PSNR and subjective quality among sophisticated algorithms in the literature.