Motion estimation (ME) for the high efficiency video coding (HEVC) occupies the largest part in the computational complexity of the HM encoder, which consists of integer ME (iME) and fractional ME (fME). We propose a fast fME algorithm that reduces its complexity substantially by reducing the number of search points to less than a half, based on statistical observations, in which four search points are selected only in the horizontal and vertical directions for half-pel search and three search points that have the smaller absolute value of the motion vector difference are selected from eight candidates for quarter-pel search. The effective number of search points in the proposed fME algorithm is reduced to about 60% and the interpolation filtering is also reduced to about 55%, compared to that of the fME in the HM encoder while experimental results show that its Bj⊘ntegaard delta (BD) rate degradation is 0.93% for the five class-B video sequences.