This paper presents a disturbance observer based controller for a quadrotor subjects to external disturbances and un-modeled dynamics. The novel control strategy is based on the hierarchial inner-outer loop structure. During the position loop controller designed, the controller is designed with an auxiliary dynamic system to achieve the position tracking, the appropriate parameters selections criteria can warrant the command attitude derivation from the position controller without singularity. During the attitude loop controller designed, unit-quaternion is used to describe attitude kinematics. In addition, a second order disturbance observer is employed to estimate and compensate the modeling uncertainties. Simulation results are given to demonstrate the effectiveness of the presented control algorithm.