Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance

This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.