High-order compact finite difference scheme for solving one-dimensional fractional diffusion equation is considered in this paper. After approximating the second-order derivative with respect to space by the compact finite difference, we use the Grünwald–Letnikov discretization of the Riemann–Liouville derivative to obtain a fully discrete implicit scheme. We analyze the local truncation error and discuss the stability using the Fourier method, then we prove that the compact finite difference scheme converges with the spatial accuracy of fourth order using matrix analysis. Numerical results are provided to verify the accuracy and efficiency of the proposed algorithm.