Energy harvesting technology has become a promising solution to enhance the energy efficiency and reduce carbon emissions in future wireless systems. In future wireless systems, most of the data throughput will come from delay-sensitive applications. To ensure a good experience for an end user, we target the optimization of the delay performance of an EH wireless system with finite energy storage. As such, it is necessary to adapt the resource allocation to the channel fading information, data queue length, and energy queue length information. The channel fading information provides the channel quality, the data queue length information provides the dynamic urgency of the transmitted data flows, and the energy queue length information provides the information on how much available energy is left in the energy buffer. Such a problem is quite challenging because it belongs to an infinite dimensional stochastic optimization. In this article, we review the existing works on the resource allocation problem in EH wireless systems. We also propose a low-complexity delay-sensitive resource control scheme and discuss valuable design insights.