Direct heuristic dynamic programming with augmented states

Jian Sun, Feng Liu, Jennie Si, Shengwei Mei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

This paper addresses a design issue of an approximate dynamic programming structure and its respective convergence property. Specifically, we propose to impose a PID structure to the action and critic networks in the direct heuristic dynamic programming (direct HDP) online learning controller. We demonstrate that the direct HDP with such PID augmented states improves convergence speed and that it out performs the traditional PID even though the learning controller may be initialized to be like a PID. Also for the first time, by using a Lyapnov approach we show that the action and critic network weights retain the property of uniformly ultimate boundedness (UUB) under mild conditions.

Original languageEnglish (US)
Title of host publication2011 International Joint Conference on Neural Networks, IJCNN 2011 - Final Program
Pages3112-3119
Number of pages8
DOIs
StatePublished - Oct 24 2011
Event2011 International Joint Conference on Neural Network, IJCNN 2011 - San Jose, CA, United States
Duration: Jul 31 2011Aug 5 2011

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Other

Other2011 International Joint Conference on Neural Network, IJCNN 2011
CountryUnited States
CitySan Jose, CA
Period7/31/118/5/11

    Fingerprint

Keywords

  • Approximate Dynamic Programming (ADP)
  • Direct Heuristic Dynamic Programming (direct HDP)
  • Feedforward Neural Network with Augmented states (AFNN)

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Sun, J., Liu, F., Si, J., & Mei, S. (2011). Direct heuristic dynamic programming with augmented states. In 2011 International Joint Conference on Neural Networks, IJCNN 2011 - Final Program (pp. 3112-3119). [6033633] (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2011.6033633