In this letter, a 10.4-Gb/s real-time baseband orthogonal frequency division multiplexing (OFDM) receiver utilizing 256/64/16-quadrature amplitude modulation (QAM) adaptive modulation format, incorporating several key functionalities, including symbol synchronization, channel equalization, as well as sampling clock frequency synchronization is successfully implemented using a commercial field programmable gate array chip and a sample rate of 5-GS/s analog-to-digital converter. The performance of the real-time baseband OFDM receiver is experimentally investigated in an intensity-modulation and direct-detection optical communication system. The off-line generated 10.4-Gb/s adaptively modulated 256/64/16QAM encoded optical OFDM signal with a high spectral efficiency of 4.84 bit/s/Hz can be successful transmitted over 20-km standard single mode fiber with a bit error rate \(< 3.8 \times 10^{\mathrm {\mathbf {-3}}}\) . In addition, the frame miss probability is real-time measured in the receiver to obtain an optimal threshold value for the symbol synchronization.