This paper describes a direct-conversion E-band transmitter (TX) in 40 nm bulk CMOS. As millimeter-wave (mm-Wave) circuits are vulnerable to process variations including the mismatch between interconnects, this E-band TX design is conducted in a layout floor-plan oriented way. Compact and symmetrical floor-plans of poly-phase filter (PPF) and I/Q modulator are presented in this work to suppress the LO feed-through (LOFT) and I/Q imbalance over both 71–76 and 81–86 GHz bands. In addition, a systematic design methodology is proposed for the mm-Wave PPF to reduce required EM simulations. The calibration methods used in this design are easy to implement for further reducing LOFT and I/Q imbalance with negligible impact on other TX performance metrics. The 0.225 mm E-band TX achieves a measured output power of 12 dBm and efficiency of 15% with about 15 GHz bandwidth. It features an uncalibrated I/Q imbalance of less than 30 dB from 62.5 to 85.5 GHz. The calibration circuits further reduce the I/Q imbalance by about 4 dB and ensure the LOFT of less than 30 dBc over more than 30 dB output dynamic range. The presented TX achieves 4.5 Gb/s 64-QAM and 14 Gb/s 16-QAM.