Easily trapped in local minima is one of the well-known problems in search point pattern based fast block motion estimation algorithms. This problem is especially serious in one-at-a-time search (OTS) and block-based gradient descent search (BBGDS). These two algorithms can provide very high speedup ratio but with low robustness in prediction accuracy especially for sequences with complex motions. Multi-path search (MPS) using more than one path have been proposed to improve the robustness of BBGDS, but the computational requirement is much increased. To tackle this problem, a novel multi-directional gradient descent search (MDGDS) is proposed in this paper with use of multiple OTSs in eight directions. Basically, the proposed MDGDS performs eight one-dimensional gradient descent searches on the error surface and therefore can trace to the global minimum more efficiently. Experimental results show that a significant improvement in computation reduction can be achieved as compared with well-known fast block motion estimation algorithms.