Performance analysis of direct heuristic dynamic programming using control-theoretic measures

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Approximate dynamic programming (ADP) has been widely studied from several important perspectives: algorithm development, learning efficiency measured by success or failure statistics, convergence rate, and learning error bounds. Given that many learning benchmarks used in ADP or reinforcement learning studies are control problems, it is important and necessary to examine the learning controllers from a control-theoretic perspective. This paper makes use of direct heuristic dynamic programming (direct HDP) and several benchmark examples to introduce a unique analytical framework that can be extended to other learning control paradigms and other complex control problems. The sensitivity analysis and the linear quadratic regulator (LQR) design are used in the paper for two purposes: to gauge direct HDP performance characteristics and to provide guidance toward designing better learning controllers. This gauge however does not limit the direct HDP to be effective only as a linear controller. Toward this end, applications of the direct HDP for nonlinear control problems beyond sensitivity analysis and the confines of LQR have been developed and compared with LQR design for command following and internal system parameter changes.

Original languageEnglish (US)
Title of host publicationThe 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings
Pages2504-2509
Number of pages6
DOIs
StatePublished - Dec 1 2007
Event2007 International Joint Conference on Neural Networks, IJCNN 2007 - Orlando, FL, United States
Duration: Aug 12 2007Aug 17 2007

Publication series

NameIEEE International Conference on Neural Networks - Conference Proceedings
ISSN (Print)1098-7576

Other

Other2007 International Joint Conference on Neural Networks, IJCNN 2007
CountryUnited States
CityOrlando, FL
Period8/12/078/17/07

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'Performance analysis of direct heuristic dynamic programming using control-theoretic measures'. Together they form a unique fingerprint.

  • Cite this

    Yang, L., Si, J., Tsakalis, K., & Rodriguez, A. (2007). Performance analysis of direct heuristic dynamic programming using control-theoretic measures. In The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings (pp. 2504-2509). [4371352] (IEEE International Conference on Neural Networks - Conference Proceedings). https://doi.org/10.1109/IJCNN.2007.4371352