The main objective of this paper is to focus on the study of limit cycle characteristics due to backlash nonlinearity in drive system. For a two-mass system, the existence of gear reduction mechanism or ball screw will introduce transmission backlash inevitably, which can aggravate mechanical resonance. Therefore, it is essential to study the limit cycle characteristics, so as to provide a good theoretical basis for limit cycle suppression. Firstly, based on describing function analysis, the system is separated into linear part and nonlinear part in order to facilitate the analysis. Then the stability and oscillation frequency of limit cycle are predicted. Through the study, it is found that the increasing of backlash amplitude does not affect the limit cycle frequency, yet it will aggravate the oscillation amplitude. Moreover, the limit cycle frequency and amplitude will increase with the increasing of system control stiffness. Finally, the theoretical analyses are confirmed by the experimental results.