An integrated design for intensified direct heuristic dynamic programming

Xiong Luo, Jennie Si, Yuchao Zhou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

There has been a growing interest in the study of adaptive/approximate dynamic programming (ADP) in recent years. The ADP technique provides a powerful tool to understand and improve the principled technologies of machine intelligence system. As one of the ADP algorithms based on adaptive critic neural networks (NNs), the direct heuristic dynamic programming (direct HDP) has demonstrated some successful applications in solving realistic engineering control problems. In this study, based on a three-network architecture in which the reinforcement signal is approximated by an additional NN, a novel integrated design method for intensified direct HDP is developed. The new design approach is implemented by using multiple PID neural networks (PIDNNs), which effectively takes into account structural knowledge of system states and control that are usually present in a physical system. By using a Lyapunov stability approach, a uniformly ultimately boundedness (UUB) result is proved for our PIDNNs-based intensified direct HDP learning controller. Furthermore, the learning and control performances of the proposed design is tested using the popular cart-pole example to illustrate the key ideas of this paper.

Original languageEnglish (US)
Title of host publicationProceedings of the 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013 - 2013 IEEE Symposium Series on Computational Intelligence, SSCI 2013
Pages183-190
Number of pages8
DOIs
StatePublished - Dec 1 2013
Event2013 4th IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013 - Singapore, Singapore
Duration: Apr 16 2013Apr 19 2013

Publication series

NameIEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL
ISSN (Print)2325-1824
ISSN (Electronic)2325-1867

Other

Other2013 4th IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013
CountrySingapore
CitySingapore
Period4/16/134/19/13

Keywords

  • Direct heuristic dynamic programming
  • PID neural network
  • neural network
  • stability

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'An integrated design for intensified direct heuristic dynamic programming'. Together they form a unique fingerprint.

  • Cite this

    Luo, X., Si, J., & Zhou, Y. (2013). An integrated design for intensified direct heuristic dynamic programming. In Proceedings of the 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013 - 2013 IEEE Symposium Series on Computational Intelligence, SSCI 2013 (pp. 183-190). [6615006] (IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL). https://doi.org/10.1109/ADPRL.2013.6615006