Abstract

This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.

Original languageEnglish (US)
Title of host publicationReinforcement Learning and Approximate Dynamic Programming for Feedback Control
PublisherJohn Wiley and Sons
Pages182-202
Number of pages21
ISBN (Print)9781118104200
DOIs
StatePublished - Feb 7 2013

Fingerprint

Dynamic programming
Controllers
Nonlinear control systems
Helicopters
Scalability
Damping
Systems analysis
Neural networks

Keywords

  • Adaptive critique, control output/value
  • ADP, reducing DP curse of dimensionality
  • Direct HDP, sensitivity maps in action/critic
  • Nonlinear ADP, performance assurance
  • Nonlinear control design using ADP

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Si, J., Yang, L., Lu, C., Tsakalis, K., & Rodriguez, A. (2013). Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance. In Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (pp. 182-202). John Wiley and Sons. https://doi.org/10.1002/9781118453988.ch9

Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance. / Si, Jennie; Yang, Lei; Lu, Chao; Tsakalis, Konstantinos; Rodriguez, Armando.

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. John Wiley and Sons, 2013. p. 182-202.

Research output: Chapter in Book/Report/Conference proceedingChapter

Si, J, Yang, L, Lu, C, Tsakalis, K & Rodriguez, A 2013, Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance. in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. John Wiley and Sons, pp. 182-202. https://doi.org/10.1002/9781118453988.ch9
Si J, Yang L, Lu C, Tsakalis K, Rodriguez A. Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance. In Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. John Wiley and Sons. 2013. p. 182-202 https://doi.org/10.1002/9781118453988.ch9
Si, Jennie ; Yang, Lei ; Lu, Chao ; Tsakalis, Konstantinos ; Rodriguez, Armando. / Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. John Wiley and Sons, 2013. pp. 182-202
@inbook{3d6c7ef0b5a64310b00ee8c96de4a313,
title = "Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance",
abstract = "This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.",
keywords = "Adaptive critique, control output/value, ADP, reducing DP curse of dimensionality, Direct HDP, sensitivity maps in action/critic, Nonlinear ADP, performance assurance, Nonlinear control design using ADP",
author = "Jennie Si and Lei Yang and Chao Lu and Konstantinos Tsakalis and Armando Rodriguez",
year = "2013",
month = "2",
day = "7",
doi = "10.1002/9781118453988.ch9",
language = "English (US)",
isbn = "9781118104200",
pages = "182--202",
booktitle = "Reinforcement Learning and Approximate Dynamic Programming for Feedback Control",
publisher = "John Wiley and Sons",

}

TY - CHAP

T1 - Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance

AU - Si, Jennie

AU - Yang, Lei

AU - Lu, Chao

AU - Tsakalis, Konstantinos

AU - Rodriguez, Armando

PY - 2013/2/7

Y1 - 2013/2/7

N2 - This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.

AB - This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.

KW - Adaptive critique, control output/value

KW - ADP, reducing DP curse of dimensionality

KW - Direct HDP, sensitivity maps in action/critic

KW - Nonlinear ADP, performance assurance

KW - Nonlinear control design using ADP

UR - http://www.scopus.com/inward/record.url?scp=84886338359&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84886338359&partnerID=8YFLogxK

U2 - 10.1002/9781118453988.ch9

DO - 10.1002/9781118453988.ch9

M3 - Chapter

AN - SCOPUS:84886338359

SN - 9781118104200

SP - 182

EP - 202

BT - Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

PB - John Wiley and Sons

ER -