To achieve accurate moving target tracking in the presence of multiple disturbances, this paper presents a desired compensation robust integral of the sign of the error (DCRISE) feedback control strategy for quadrotor unmanned aerial vehicles under image-based visual servo (IBVS) framework. For the purpose of effectively regulating the pose and attitude of quadrotor, proper perspective image moments extracted from the defined virtual image plane are chosen as the image features. Then, a DCRISE feedback controller is constructed for the quadrotor, in which a robust integral of the sign of the error method is employed for suppressing the multiple disturbances and a desired compensation-based feedforward control is introduced to address the nonlinearity nature of the system and circumvent the requirements on the availability of linear velocity. It is theoretically proved by the Lyapunov analysis that all the closed-loop tracking errors under the proposed DCRISE-based IBVS control structure are convergent to a small neighborhood around the origin in spite of multiple disturbances. The validity and advantages of the present approach are confirmed through extensive simulations and comparisons.