Summary
In this chapter a new online direct adaptive scheme is presented which converges to the optimal state feedback control solution for affine in the inputs nonlinear systems. The optimal control solution is obtained in a direct fashion, without system identification. The optimal adaptive control algorithm is derived in a continuous-time framework. The algorithm is an online approach to policy iterations based on an adaptive critic structure to find an approximate solution to the state feedback, infinite-horizon, optimal control problem.