An integrated design for intensified direct heuristic dynamic programming

Xiong Luo, Jennie Si, Yuchao Zhou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

There has been a growing interest in the study of adaptive/approximate dynamic programming (ADP) in recent years. The ADP technique provides a powerful tool to understand and improve the principled technologies of machine intelligence system. As one of the ADP algorithms based on adaptive critic neural networks (NNs), the direct heuristic dynamic programming (direct HDP) has demonstrated some successful applications in solving realistic engineering control problems. In this study, based on a three-network architecture in which the reinforcement signal is approximated by an additional NN, a novel integrated design method for intensified direct HDP is developed. The new design approach is implemented by using multiple PID neural networks (PIDNNs), which effectively takes into account structural knowledge of system states and control that are usually present in a physical system. By using a Lyapunov stability approach, a uniformly ultimately boundedness (UUB) result is proved for our PIDNNs-based intensified direct HDP learning controller. Furthermore, the learning and control performances of the proposed design is tested using the popular cart-pole example to illustrate the key ideas of this paper.

Original languageEnglish (US)
Title of host publicationIEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL
Pages183-190
Number of pages8
DOIs
StatePublished - 2013
Event2013 4th IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013 - Singapore, Singapore
Duration: Apr 16 2013Apr 19 2013

Other

Other2013 4th IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013
CountrySingapore
CitySingapore
Period4/16/134/19/13

Fingerprint

Dynamic programming
Neural networks
Network architecture
Poles
Reinforcement
Controllers

Keywords

  • Direct heuristic dynamic programming
  • neural network
  • PID neural network
  • stability

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software

Cite this

Luo, X., Si, J., & Zhou, Y. (2013). An integrated design for intensified direct heuristic dynamic programming. In IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL (pp. 183-190). [6615006] https://doi.org/10.1109/ADPRL.2013.6615006

An integrated design for intensified direct heuristic dynamic programming. / Luo, Xiong; Si, Jennie; Zhou, Yuchao.

IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL. 2013. p. 183-190 6615006.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Luo, X, Si, J & Zhou, Y 2013, An integrated design for intensified direct heuristic dynamic programming. in IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL., 6615006, pp. 183-190, 2013 4th IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2013, Singapore, Singapore, 4/16/13. https://doi.org/10.1109/ADPRL.2013.6615006
Luo X, Si J, Zhou Y. An integrated design for intensified direct heuristic dynamic programming. In IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL. 2013. p. 183-190. 6615006 https://doi.org/10.1109/ADPRL.2013.6615006
Luo, Xiong ; Si, Jennie ; Zhou, Yuchao. / An integrated design for intensified direct heuristic dynamic programming. IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL. 2013. pp. 183-190
@inproceedings{b014d61b9a4243d79771e38526944721,
title = "An integrated design for intensified direct heuristic dynamic programming",
abstract = "There has been a growing interest in the study of adaptive/approximate dynamic programming (ADP) in recent years. The ADP technique provides a powerful tool to understand and improve the principled technologies of machine intelligence system. As one of the ADP algorithms based on adaptive critic neural networks (NNs), the direct heuristic dynamic programming (direct HDP) has demonstrated some successful applications in solving realistic engineering control problems. In this study, based on a three-network architecture in which the reinforcement signal is approximated by an additional NN, a novel integrated design method for intensified direct HDP is developed. The new design approach is implemented by using multiple PID neural networks (PIDNNs), which effectively takes into account structural knowledge of system states and control that are usually present in a physical system. By using a Lyapunov stability approach, a uniformly ultimately boundedness (UUB) result is proved for our PIDNNs-based intensified direct HDP learning controller. Furthermore, the learning and control performances of the proposed design is tested using the popular cart-pole example to illustrate the key ideas of this paper.",
keywords = "Direct heuristic dynamic programming, neural network, PID neural network, stability",
author = "Xiong Luo and Jennie Si and Yuchao Zhou",
year = "2013",
doi = "10.1109/ADPRL.2013.6615006",
language = "English (US)",
isbn = "9781467359252",
pages = "183--190",
booktitle = "IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL",

}

TY - GEN

T1 - An integrated design for intensified direct heuristic dynamic programming

AU - Luo, Xiong

AU - Si, Jennie

AU - Zhou, Yuchao

PY - 2013

Y1 - 2013

N2 - There has been a growing interest in the study of adaptive/approximate dynamic programming (ADP) in recent years. The ADP technique provides a powerful tool to understand and improve the principled technologies of machine intelligence system. As one of the ADP algorithms based on adaptive critic neural networks (NNs), the direct heuristic dynamic programming (direct HDP) has demonstrated some successful applications in solving realistic engineering control problems. In this study, based on a three-network architecture in which the reinforcement signal is approximated by an additional NN, a novel integrated design method for intensified direct HDP is developed. The new design approach is implemented by using multiple PID neural networks (PIDNNs), which effectively takes into account structural knowledge of system states and control that are usually present in a physical system. By using a Lyapunov stability approach, a uniformly ultimately boundedness (UUB) result is proved for our PIDNNs-based intensified direct HDP learning controller. Furthermore, the learning and control performances of the proposed design is tested using the popular cart-pole example to illustrate the key ideas of this paper.

AB - There has been a growing interest in the study of adaptive/approximate dynamic programming (ADP) in recent years. The ADP technique provides a powerful tool to understand and improve the principled technologies of machine intelligence system. As one of the ADP algorithms based on adaptive critic neural networks (NNs), the direct heuristic dynamic programming (direct HDP) has demonstrated some successful applications in solving realistic engineering control problems. In this study, based on a three-network architecture in which the reinforcement signal is approximated by an additional NN, a novel integrated design method for intensified direct HDP is developed. The new design approach is implemented by using multiple PID neural networks (PIDNNs), which effectively takes into account structural knowledge of system states and control that are usually present in a physical system. By using a Lyapunov stability approach, a uniformly ultimately boundedness (UUB) result is proved for our PIDNNs-based intensified direct HDP learning controller. Furthermore, the learning and control performances of the proposed design is tested using the popular cart-pole example to illustrate the key ideas of this paper.

KW - Direct heuristic dynamic programming

KW - neural network

KW - PID neural network

KW - stability

UR - http://www.scopus.com/inward/record.url?scp=84891522328&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891522328&partnerID=8YFLogxK

U2 - 10.1109/ADPRL.2013.6615006

DO - 10.1109/ADPRL.2013.6615006

M3 - Conference contribution

AN - SCOPUS:84891522328

SN - 9781467359252

SP - 183

EP - 190

BT - IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL

ER -