In this paper, we investigate the consensus problem for the leader-follower multi-agent systems with second-order dynamics subject to external disturbances. Based on integral sliding mode (ISM) control technique, we design a new sliding mode surface to achieve consensus of multi-agent system subjected to external disturbance. With the help of control protocol, system trajectory may reach the sliding mode surface in finite time. Under the case of one leader, the states of followers may track that of the leader to achieve consensus. Moreover, the states of system asymptotically converge to convex hull spanned by that of multiple leaders. Finally, we give simulation results to verify effectiveness of our theory.