We experimentally demonstrate direct generation of 51 nJ, 284-fs ultrashort pulses in an all-fiber cavity-optimized erbium-doped laser with a low repetition rate of 2.68 MHz. The conversion efficiency of the laser is about 10%. The laser is mode-locked by nonlinear polarization rotation. Such high energy and ultrashort duration benefits from using large output ratio (90%), elongating the cavity length, dispersion management, and bidirectional high-power pumping in the laser. Further decreasing the net dispersion of the laser by using a dispersion-compensated fiber can reduce the pulse duration to 160 fs. In addition, variation of spectral width due to an invisible filter in the laser is experimentally demonstrated.