With the development of smart grid technology, the user participation in demand side gradually plays a more important role in energy management. Because of the uncertainties of user behavior, it is not easy to accurately obtain the complete information for energy management in smart grids. What's more, energy users have different respond to the energy management strategy. The competition of energy users with different priorities is regard as a zero-sum game. The fuzzy Markov game energy management controller is proposed to deal with the presence of unknown parameter variations. The problems of parameter optimization are solved by simulated annealing algorithm. The proposed controller can learn to take the best action to regulate energy usage for different users. Simulation results demonstrate the viability of the proposed controller for accelerating learning and promoting the performance of energy usage in smart grids.