Direct heuristic dynamic programming with augmented states

Jian Sun, Feng Liu, Jennie Si, Shengwei Mei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

This paper addresses a design issue of an approximate dynamic programming structure and its respective convergence property. Specifically, we propose to impose a PID structure to the action and critic networks in the direct heuristic dynamic programming (direct HDP) online learning controller. We demonstrate that the direct HDP with such PID augmented states improves convergence speed and that it out performs the traditional PID even though the learning controller may be initialized to be like a PID. Also for the first time, by using a Lyapnov approach we show that the action and critic network weights retain the property of uniformly ultimate boundedness (UUB) under mild conditions.

Original languageEnglish (US)
Title of host publicationProceedings of the International Joint Conference on Neural Networks
Pages3112-3119
Number of pages8
DOIs
StatePublished - 2011
Event2011 International Joint Conference on Neural Network, IJCNN 2011 - San Jose, CA, United States
Duration: Jul 31 2011Aug 5 2011

Other

Other2011 International Joint Conference on Neural Network, IJCNN 2011
CountryUnited States
CitySan Jose, CA
Period7/31/118/5/11

Fingerprint

Dynamic programming
Controllers

Keywords

  • Approximate Dynamic Programming (ADP)
  • Direct Heuristic Dynamic Programming (direct HDP)
  • Feedforward Neural Network with Augmented states (AFNN)

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Sun, J., Liu, F., Si, J., & Mei, S. (2011). Direct heuristic dynamic programming with augmented states. In Proceedings of the International Joint Conference on Neural Networks (pp. 3112-3119). [6033633] https://doi.org/10.1109/IJCNN.2011.6033633

Direct heuristic dynamic programming with augmented states. / Sun, Jian; Liu, Feng; Si, Jennie; Mei, Shengwei.

Proceedings of the International Joint Conference on Neural Networks. 2011. p. 3112-3119 6033633.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, J, Liu, F, Si, J & Mei, S 2011, Direct heuristic dynamic programming with augmented states. in Proceedings of the International Joint Conference on Neural Networks., 6033633, pp. 3112-3119, 2011 International Joint Conference on Neural Network, IJCNN 2011, San Jose, CA, United States, 7/31/11. https://doi.org/10.1109/IJCNN.2011.6033633
Sun J, Liu F, Si J, Mei S. Direct heuristic dynamic programming with augmented states. In Proceedings of the International Joint Conference on Neural Networks. 2011. p. 3112-3119. 6033633 https://doi.org/10.1109/IJCNN.2011.6033633
Sun, Jian ; Liu, Feng ; Si, Jennie ; Mei, Shengwei. / Direct heuristic dynamic programming with augmented states. Proceedings of the International Joint Conference on Neural Networks. 2011. pp. 3112-3119
@inproceedings{25e1ab6b5bdd4f108aab1e6d48281d1d,
title = "Direct heuristic dynamic programming with augmented states",
abstract = "This paper addresses a design issue of an approximate dynamic programming structure and its respective convergence property. Specifically, we propose to impose a PID structure to the action and critic networks in the direct heuristic dynamic programming (direct HDP) online learning controller. We demonstrate that the direct HDP with such PID augmented states improves convergence speed and that it out performs the traditional PID even though the learning controller may be initialized to be like a PID. Also for the first time, by using a Lyapnov approach we show that the action and critic network weights retain the property of uniformly ultimate boundedness (UUB) under mild conditions.",
keywords = "Approximate Dynamic Programming (ADP), Direct Heuristic Dynamic Programming (direct HDP), Feedforward Neural Network with Augmented states (AFNN)",
author = "Jian Sun and Feng Liu and Jennie Si and Shengwei Mei",
year = "2011",
doi = "10.1109/IJCNN.2011.6033633",
language = "English (US)",
isbn = "9781457710865",
pages = "3112--3119",
booktitle = "Proceedings of the International Joint Conference on Neural Networks",

}

TY - GEN

T1 - Direct heuristic dynamic programming with augmented states

AU - Sun, Jian

AU - Liu, Feng

AU - Si, Jennie

AU - Mei, Shengwei

PY - 2011

Y1 - 2011

N2 - This paper addresses a design issue of an approximate dynamic programming structure and its respective convergence property. Specifically, we propose to impose a PID structure to the action and critic networks in the direct heuristic dynamic programming (direct HDP) online learning controller. We demonstrate that the direct HDP with such PID augmented states improves convergence speed and that it out performs the traditional PID even though the learning controller may be initialized to be like a PID. Also for the first time, by using a Lyapnov approach we show that the action and critic network weights retain the property of uniformly ultimate boundedness (UUB) under mild conditions.

AB - This paper addresses a design issue of an approximate dynamic programming structure and its respective convergence property. Specifically, we propose to impose a PID structure to the action and critic networks in the direct heuristic dynamic programming (direct HDP) online learning controller. We demonstrate that the direct HDP with such PID augmented states improves convergence speed and that it out performs the traditional PID even though the learning controller may be initialized to be like a PID. Also for the first time, by using a Lyapnov approach we show that the action and critic network weights retain the property of uniformly ultimate boundedness (UUB) under mild conditions.

KW - Approximate Dynamic Programming (ADP)

KW - Direct Heuristic Dynamic Programming (direct HDP)

KW - Feedforward Neural Network with Augmented states (AFNN)

UR - http://www.scopus.com/inward/record.url?scp=80054769521&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80054769521&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2011.6033633

DO - 10.1109/IJCNN.2011.6033633

M3 - Conference contribution

AN - SCOPUS:80054769521

SN - 9781457710865

SP - 3112

EP - 3119

BT - Proceedings of the International Joint Conference on Neural Networks

ER -