In generating a digital elevation map from an interferogram obtained by interferometric synthetic aperture radar, the filtering process is as important as the phase unwrapping process. Before unwrapping, we usually have to restore the data image by reducing so-called singular points (SPs) by filtering process without destroying or smearing delicate fringes. Previously, an effective SP restoration method was proposed based on the complex-valued Markov random field (CMRF) model. However, there is still room for improvement in the definition of SPs and in the formulation of CMRF parameter estimation. In this letter, we propose a novel scheme by introducing a new concept, namely, the ldquosingular unit.rdquo We also estimate CMRF parameters locally as a weighted sum by taking the distance and SP numbers in sample sites into account. By using this restoration method, we demonstrate a high-performance removal of SPs. We also find that delicate landscape features, which are often lost in conventional filtering, are preserved appropriately. We confirm the quality in higher signal-to-noise ratios obtained against actual height data.