As one of the major radio-frequency impairments in multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) system, in-phase and quadrature-phase (IQ) imbalance tends to destroy the reciprocity between uplink and downlink channel based on time-division duplex (TDD) mode and deteriorate system performance. In this paper, a time-domain calibration scheme is proposed to restore the channel reciprocity for TDD MIMO-OFDM system in the presence of frequency-independent IQ imbalance. The calibration model is built and two calibration matrices are designed to precode the uplink and downlink transmission, respectively. Complexity analysis shows that the proposed scheme is with low computational complexity, which is linear in the number of antennas at the base station and mobile station. Simulation results show that the proposed scheme can restore the channel reciprocity and provide the performance of bit error rate and cumulative distribution functions of downlink capacity close to the ideal case without IQ imbalance.