As a novel two-stage greedy approximation algorithm, Forward-Backward Pursuit (FBP) algorithm attracts wide attention because of its high reconstruction accuracy and no need for sparsity as a priori information. However, the FBP algorithm has to spend much more time to get a higher accuracy. In view of this, an Improved Adaptive Forward-Backward Matching Pursuit (IAFBP) algorithm is proposed in this paper. In the forward stage, the IAFBP algorithm uses an adaptive threshold to select the appropriate number of atoms into support set, so that the number of selected atoms is more random. In the backward stage, the projection coefficient of the atoms is taken as the basis of rejection, and the deletion threshold is introduced to reject the atoms adaptively, so that more right atoms are retained in each iteration and the reconstruction speed can be accelerated. At the same time, it overcomes the excessive backtracking phenomenon existing in the adaptive process and improves the accuracy of the algorithm. The simulation results of one-dimensional sparse signals and two-dimensional images show that the IAFBP algorithm has more advantages than the FBP algorithm in reconstruction performance and computational time.