In this paper, we investigate underlay device-to-device (D2D) communication in cellular networks where D2D transmitters transmit data to their receivers using cellular spectrum. We analyze the average sum throughput of D2D receivers in interference-limited channels. To enhance it, we propose a semi-distributed spectrum access scheme, which consists of two stages. In the first stage, the cellular base station divides whole D2D transmitters into multiple groups and assigns a different subchannel to each group. In the second stage, D2D transmitters in each group randomly access the subchannel assigned to it with predetermined access probability. We formulate an optimization problem to find groups and access probabilities, which maximize the average sum throughput. To overcome the prohibitive computational complexity to obtain its optimal solution, we decompose it into two subproblems: one to find groups and one to find access probabilities. A heuristic grouping algorithm is adopted to solve the former, and a branch-and-bound-based algorithm is proposed to solve the latter with achieving a target accuracy. The validity of the branch-and-bound-based algorithm is shown by performance comparison with an exhaustive search. It is shown that the heuristic grouping and the branch-and-bound-based algorithm achieve higher average sum throughput than conventional methods.