In this paper, a new PolSAR speckle filtering approach is proposed to overcome the deficiency of SMB filter on X-band polarimetric TerraSAR data. Instead of Freeman-Durden decomposition, the new approach applies Barnes-Holm decomposition to label each pixel by dominant scattering mechanism as single point target scattering or distributed target scattering, and then a simplified Cameron classification scheme is used to separate the point target scattering dominated pixels into two dominant scattering categories: trihedral and dihedral. Then as the SMB filter, the unsupervised Wishart classification is applied to determine the adaptive neighborhood. At last, only the “N-target” component is involved in the filtering step, while the pure single target component is preserved. By experiment on San Francisco multi-look TerraSAR-X PolSAR data, it is illustrated that the new method performs better than SMB filter in pixel dominant scattering mechanism labeling and preservation of point target polarimetric properties.