It has been suggested that inverse models serve feedforward controllers in the human brain. We have proposed a novel learning scheme to acquire a neural inverse model of a controlled object. This scheme propagates error “forward” in a multi-layered neural network to solve a credit assignment problem based on Newton-like method. In this paper, we apply a RLS algorithm to this scheme for the stability of learning. The suitability of the proposed scheme was confirmed by computer simulation; it could acquire an inverse dynamics model of a 2-link arm faster than a conventional scheme based on a back-propagation rule.