M-ary Elliptical Phase Shift Keying (M-EPSK) have been proposed and evaluated through simulations in our previous works. According to the simulation results, M-EPSK (M=4,8) can achieve better noise performance than MPSK; in particular, M-EPSK can adaptively realize trade-offs between bandwidth efficiency and power efficiency by varying the eccentricity. Eccentricity is the main modulation variable that makes M-EPSK distinctive comparing with conventional MPSK. Substantial effects of eccentricity to performance of M-EPSK have been verified in simulations; however, prior to exploring applications of M-EPSK in communication systems, principle behind the effects of eccentricity to performance needs to be further studied. In this paper, we theoretically evaluate effects of eccentricity to decision parameters of performance, and then explore a novel solution based on approximate treatment to derive closed-form BER approximations of M-EPSK over an AWGN channel. In the BER approximations, the effects of eccentricity to both error performance and bandwidth efficiency of M-EPSK are mathematically derived. In addition, the theoretical results agree with simulation results.