Reducing the head positioning error is important to achieve higher track density in hard disk drives. In this paper, it is shown that the head off-track due to disk vibration can be reduced by using a head gimbal assembly capable of moving not only vertically to disk surface but also radially across the track. We find the optimal geometric relationship to minimize the head off-track due to disk vibration. The relationship is obtained based on precise mathematical modeling of head off-track mechanism due to disk vibration. Some examples of head gimbal assembly (HGA) with radial head motion capability, which satisfy such optimal relationship, are also proposed. It is experimentally found that the proposed optimal HGA can reduce non-repeatable run-out (NRRO) position error signal (PES) significantly. Since it reduces NRRO PES during servo track writing as well, the written-in portion of repeatable run-out PES can be also significantly reduced.