The Long Term Evolution (LTE) adopts the femtocell technology (i.e., indoor base station; BS) to offload data traffic from the macrocell (i.e., outdoor BS) and extend the radio coverage of the macrocell. In the LTE femtocell/macrocell network, it is more likely for a mobile station (MS) to perform the handover procedure to switch the RRC connection between femtocells and macrocells. In this paper, we propose a Reducing Handover Cost (RHC) mechanism to achieve two goals: to avoid frequent handover executions and to increase the traffic offloading capability of femtocells. RHC is considered practical and has been implemented in the femtocell prototype. Based on the analytical models and the simulation experiments, we investigate the performance of the RHC mechanism. Our study shows that the RHC mechanism can effectively achieve the two goals.