We review the evolution of modulation formats designed for direct detection, from on-off keying intensity modulation (IM) of a single polarization to three-dimensional formats where the intensity of both polarizations and the interpolarization phase are all independently modulated. We present the transmitter and receiver architectures capable of modulating an increasing number of dimensions, all orthogonal to each other. We detail the digital signal processing for Stokes vector receivers capable of recovering all three dimensions of a lightwave via self-beating direct detection, performing simultaneously polarization derotation and mitigation of intersymbol interference. We also analyze two different transmitter architectures modulating all three dimensions where the interpolarization phase is modulated over two levels using either a direct phase modulator (DPM) or a binary phase modulator, also called a BPSK modulator. We study the impact of increasing the bandwidth efficiency from 4 to 5 b/symbol by adding two level modulation of the interpolarization phase of a dual-polarization 4-level pulse amplitude modulation (PAM4) signal on the signal quality of the PAM4 IM on the polarization bearing the phase modulator by modeling the optical fields generated by both architectures. Assuming equal quality of the driving signals to the PAM4 and phase modulator, we show that a DPM minimizes the impact on PAM4 for short reach direct detect systems operating close to the zero dispersion wavelength where chromatic dispersion is negligible.