In this paper, we propose a mechanism for mode selection and spectrum allocation for in-band overlay D2D com-munication. A potential D2D user measures the activity over the spectrum allocated for D2D transmissions and uses a carrier sensing threshold to decide about its transmission mode. By appropriately selecting the carrier sensing threshold, the interference among D2D communication pairs can be controlled and their performance can be improved. Also, the distributed nature of this mechanism leads to less signalling overhead between D2D users and base stations even in dense deployments. Based on this method, we find spectrum allocation factors and carrier sensing thresholds for maximizing the rate of D2D users under target rate constraint for cellular users.