Direct heuristic dynamic programming based on an improved PID neural network and initial weighs choosing method

Jian Sun, Feng Liu, Jennie Si, Shengwei Mei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As an online learning algorithm of approximate dynamic programming (ADP), direct heuristic dynamic programming (DHDP) has demonstrated its applicability to large state and control problems. However, there still lacks of a systemic approach to initialize the network weights for DHDP. In this paper, an improved PID-neural network (IPIDNN) configuration is proposed and applied to the critic and action networks of DHDP, which is flexible and easy to expand. Because of incorporating an inherent PID control structure, it is easy to use a well-designed PID controller to guide the initial weighs choosing for the action network. Based on this framework, a novel initializing approach is suggested based on a PID controller, such that the DHDP learning process starts from a good enough initial state. Simulations are carried on a cart-pole system to validate the effectiveness of the IPIDNN-based DHDP and the proposed initializing approach.

Original languageEnglish (US)
Title of host publication2010 5th International Conference on Critical Infrastructure, CRIS 2010 - Proceedings
DOIs
StatePublished - Dec 15 2010
Event2010 5th International Conference on Critical Infrastructure, CRIS 2010 - Beijing, China
Duration: Sep 20 2010Sep 22 2010

Publication series

Name2010 5th International Conference on Critical Infrastructure, CRIS 2010 - Proceedings

Other

Other2010 5th International Conference on Critical Infrastructure, CRIS 2010
CountryChina
CityBeijing
Period9/20/109/22/10

Keywords

  • Approximate dynamic programming (ADP)
  • Direct heuristic dynamic programming (direct HDP)
  • Improved PID neural network (IPIDNN)
  • Initial weighs choosing
  • PID controller

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Direct heuristic dynamic programming based on an improved PID neural network and initial weighs choosing method'. Together they form a unique fingerprint.

  • Cite this

    Sun, J., Liu, F., Si, J., & Mei, S. (2010). Direct heuristic dynamic programming based on an improved PID neural network and initial weighs choosing method. In 2010 5th International Conference on Critical Infrastructure, CRIS 2010 - Proceedings [5617558] (2010 5th International Conference on Critical Infrastructure, CRIS 2010 - Proceedings). https://doi.org/10.1109/CRIS.2010.5617558