This letter presents an effective approach for synthetic aperture radar (SAR) image registration. Corresponding edge features are extracted by exploring the local spatial relationship around corresponding speeded-up robust feature points. Only edge pairs that are matched in the aspect of distance and orientation to the center feature points are taken as effective edge features. For edge-point set matching, we introduce the coherent point drift (CPD) registration method. CPD is able to maintain the global topological structure of edges and is robust to outliers. We refine the original CPD by taking our confidence on edge points as prior information. A new objective function is deduced, resulting in a better registration performance. Experimental results show that the proposed method performs well in terms of registration robustness and accuracy.