This paper, for the first time, experimentally presents direct comparisons of two 25 GHz spaced wavelength division multiplexing (WDM) systems using conventional dual-polarization (DP) 16-ary quadrature amplitude modulation and duobinary-shaped DP-quadrature phase shift keying (QPSK) modulation. Both systems operate at the same bit rate per channel of 112 Gbit/s, yielding a spectral efficiency of 4.1 bit/s/Hz. The comparisons are conducted for three different cases, i.e., the back-to-back sensitivity, the nonlinear tolerance over a 640-km standard single-mode fiber link, and the phase-noise tolerance (by means of simulations). The results show that the duobinary-shaped DP-QPSK system not only provides a 3.4 dB superior back-to-back sensitivity, but also exhibits a 3 dB higher tolerance against nonlinear impairments after 640 km transmission with three WDM channels. In addition, the numerical results indicate that both investigated systems provide similar tolerances to the laser phase noise given that the block length used in the carrier phase estimation is optimized.