In this paper, we propose a novel closed-loop approach for robust downlink multi-antenna cross-layer design with outdated channel state information at the transmitter (CSIT). Based on the ACK/NAK feedbacks from the mobiles, the proposed cross-layer design does not require the knowledge of CSIT error statistics. We formulate the cross-layer design as a mixed combinatorial search and Markov decision process (MDP). While it is well-known that general solutions for MDP are very complex, one important contribution of this paper is that we obtain simple closed-form power, rate and user allocation policies which are asymptotically optimal for small target frame error rate (FER). Simulation results illustrate that the performance of the proposed closed-loop cross-layer design is very robust with respect to the outdated CSIT, CSIT error model mismatch as well as channel variations due to Doppler.