Cooperation learning is one main part of research on multi-agent system. Based on distributed reinforcement learning, a method of multi-agent coordination is proposed. By means of this method, at first a global complicated task is decomposed, and then the central reinforcement learning is adopted to coordinate and assign subtasks, and the individual reinforcement is adopted to choose the effective action. With the application and experiment in robot soccer simulation game, this method has better performance than the conventional reinforcement learning.