In this work, we propose a new framework called sequential reduction (SR) for reducing lattice bases, which harnesses the approximate closest vector problem (εCVP) oracles to sequentially reduce each basis vector. With the best εCVP oracle to serve as the theoretical upper limit of this scheme, our bound on basis lengths can be better than that of Minkowski's reduction under a mild assumption. A practical subroutine called greedy cancellation (GC) is also proposed to implement εCVP, whose reduced basis can be proved Gauss-reduced for all distinct pairs of vectors. Simulations validate the superiority of our low complexity method in the context of detection problems of large MIMO systems.