Compressed sensing for MRI (CS-MRI) attempts to recover an object from undersampled k-space data by minimizing sparsity-promoting regularization criteria. The iterative reweighted least squares (IRLS) algorithm can perform the minimization task by solving iteration-dependent linear systems, recursively. However, this process can be slow as the associated linear system is often poorly conditioned for ill-posed problems. We propose a new scheme based on the matrix inversion lemma (MIL) to accelerate the solving process. We demonstrate numerically for CS-MRI that our method provides significant speed-up compared to linear and nonlinear conjugate gradient algorithms, thus making it a promising alternative for such applications.