We consider a single source transmitting media streams to multiple users over a shared wireless channel. The channel for each user is time-varying, and each user has a buffer to store received packets before they are decoded and played. At each time step, the source determines how much power to use for transmission to each user. The objective is for the source to allocate power in a manner that minimizes an expected cost measure, while satisfying strict buffer underflow constraints and a total power constraint in each slot. The expected cost measure is composed of costs associated with power consumption from transmission and packet holding costs. The buffer underflow constraints prevent the user buffers from emptying, so as to maintain playout quality. In the case of a single user, we show that a modified base-stock policy is optimal under the finite and infinite horizon discounted expected cost criteria. We present the sequences of critical numbers that characterize the optimal control laws in each of these two problems. We also discuss the structure of the optimal policy in the multi-user case.