In this paper, we consider device-to-device (D2D) communication underlaying cellular networks, where D2D users harvest radio frequency (RF) power from uplink cellular transmissions. This paper addresses two important issues for energy harvesting-based D2D cellular networks. The first is how the energy harvested from cellular ambient RF signals affects the D2D spectral efficiency. Using tools from stochastic geometry, we investigate this issue by first characterizing the transmission probability of D2D transmitter as the probability of having enough battery power, and then obtaining analytic expressions of the spectral efficiency for D2D and cellular networks. The second issue is related to the cellular spectral efficiency: the more RF energy D2D users can harvest, the higher their transmission probability becomes, which leads to generating more interference in the cellular network. We carry out simulations to understand this impact of RF energy harvesting on the spectral efficiency of both D2D and cellular networks. Results show promising improvement to the whole network, in terms of the weighted spectral efficiency, when employing RF harvesting technology and when there are enough available channels in the network.