Voluntary participation, as a simple yet valid mechanism to promote cooperation in game theory, has been received a great deal of attention. Besides, historical payoff has also been verified to be an efficient way of promoting cooperation. Inspired by these facts, here we introduce historical payoff into the measure of fitness in voluntary prisoner's dilemma. In detail, the contribution of historical payoff is decided by the tunable parameter w: when w equals to zero, the traditional voluntary prisoner's dilemma game returns, where no historical payoff is contained; while positive w involves the impact of historical payoff. It is shown that cooperative behavior is remarkably promoted with increment of w for small b; while for large b, system will show the cycle dominance of these strategies and avoid the dominance of defection. This indicates that historical payoff has a positive impact on the evolution of cooperation.