Digital image correlation (DIC) technique has been increasingly employed to implement surface deformation measurements in many engineering fields. Practically, it has been demonstrated that the choice of subset sizes exerts a strong influence on measurement results of DIC, especially when there exists locally larger deformation over the subsets involved. This paper proposes a novel subpixel registration algorithm with Gaussian windows to implicitly optimize the subset sizes by adjusting the shape of Gaussian windows in a self-adaptive fashion with the aid of a so-called weighted zero-normalized sum-of-squared difference correlation criterion. The feasibility and effectiveness of the self-adaptive algorithm are carefully verified through a set of well-designed synthetic speckle images, which indicates that the presented algorithm is able to greatly enhance the accuracy and precision of displacement measurements as compared with the traditional subpixel registration methods.