Direct neural dynamic programming

Jennie Si, Lei Yang, Derong Liu

Research output: Chapter in Book/Report/Conference proceedingChapter

10 Citations (Scopus)

Abstract

This chapter introduces direct neural dynamic programming (direct NDP), which belongs to the class of heuristic dynamic programming algorithms discussed in Chapters 3, 4, and 19. However, direct NDP is a model-independent approach to action-dependent heuristic dynamic programming. It is, therefore, an on-line learning control paradigm. This chapter contains a comparison study using other well-known algorithms to help readers gain quantitative insight on several ADP algorithms. It also contains results of direct NDP controlling a triple-link inverted pendulum using many continuous state variables and a continuous control, and direct NDP in a wireless network call admission control application. Furthermore, in Chapter 21 direct NDP is demonstrated on an industrial scale Apache helicopter model for stabilization, tracking control, and reconfiguration after component failure. Preliminary results indicate that direct NDP has the potential to address large-scale problems.

Original languageEnglish (US)
Title of host publicationHandbook of Learning and Approximate Dynamic Programming
PublisherJohn Wiley and Sons Inc.
Pages125-151
Number of pages27
ISBN (Electronic)9780470544785
ISBN (Print)047166054X, 9780471660545
DOIs
StatePublished - Jan 1 2004

Fingerprint

Dynamic programming
Administrative data processing
Congestion control (communication)
Pendulums
Helicopters
Wireless networks
Stabilization

Keywords

  • Adaptation model
  • Approximation methods
  • Artificial neural networks
  • Dynamic programming
  • Learning
  • Learning systems
  • Markov processes

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Si, J., Yang, L., & Liu, D. (2004). Direct neural dynamic programming. In Handbook of Learning and Approximate Dynamic Programming (pp. 125-151). John Wiley and Sons Inc.. https://doi.org/10.1109/9780470544785.ch5

Direct neural dynamic programming. / Si, Jennie; Yang, Lei; Liu, Derong.

Handbook of Learning and Approximate Dynamic Programming. John Wiley and Sons Inc., 2004. p. 125-151.

Research output: Chapter in Book/Report/Conference proceedingChapter

Si, J, Yang, L & Liu, D 2004, Direct neural dynamic programming. in Handbook of Learning and Approximate Dynamic Programming. John Wiley and Sons Inc., pp. 125-151. https://doi.org/10.1109/9780470544785.ch5
Si J, Yang L, Liu D. Direct neural dynamic programming. In Handbook of Learning and Approximate Dynamic Programming. John Wiley and Sons Inc. 2004. p. 125-151 https://doi.org/10.1109/9780470544785.ch5
Si, Jennie ; Yang, Lei ; Liu, Derong. / Direct neural dynamic programming. Handbook of Learning and Approximate Dynamic Programming. John Wiley and Sons Inc., 2004. pp. 125-151
@inbook{d0edb58cf80d490dadeebd8061f1b575,
title = "Direct neural dynamic programming",
abstract = "This chapter introduces direct neural dynamic programming (direct NDP), which belongs to the class of heuristic dynamic programming algorithms discussed in Chapters 3, 4, and 19. However, direct NDP is a model-independent approach to action-dependent heuristic dynamic programming. It is, therefore, an on-line learning control paradigm. This chapter contains a comparison study using other well-known algorithms to help readers gain quantitative insight on several ADP algorithms. It also contains results of direct NDP controlling a triple-link inverted pendulum using many continuous state variables and a continuous control, and direct NDP in a wireless network call admission control application. Furthermore, in Chapter 21 direct NDP is demonstrated on an industrial scale Apache helicopter model for stabilization, tracking control, and reconfiguration after component failure. Preliminary results indicate that direct NDP has the potential to address large-scale problems.",
keywords = "Adaptation model, Approximation methods, Artificial neural networks, Dynamic programming, Learning, Learning systems, Markov processes",
author = "Jennie Si and Lei Yang and Derong Liu",
year = "2004",
month = "1",
day = "1",
doi = "10.1109/9780470544785.ch5",
language = "English (US)",
isbn = "047166054X",
pages = "125--151",
booktitle = "Handbook of Learning and Approximate Dynamic Programming",
publisher = "John Wiley and Sons Inc.",
address = "United States",

}

TY - CHAP

T1 - Direct neural dynamic programming

AU - Si, Jennie

AU - Yang, Lei

AU - Liu, Derong

PY - 2004/1/1

Y1 - 2004/1/1

N2 - This chapter introduces direct neural dynamic programming (direct NDP), which belongs to the class of heuristic dynamic programming algorithms discussed in Chapters 3, 4, and 19. However, direct NDP is a model-independent approach to action-dependent heuristic dynamic programming. It is, therefore, an on-line learning control paradigm. This chapter contains a comparison study using other well-known algorithms to help readers gain quantitative insight on several ADP algorithms. It also contains results of direct NDP controlling a triple-link inverted pendulum using many continuous state variables and a continuous control, and direct NDP in a wireless network call admission control application. Furthermore, in Chapter 21 direct NDP is demonstrated on an industrial scale Apache helicopter model for stabilization, tracking control, and reconfiguration after component failure. Preliminary results indicate that direct NDP has the potential to address large-scale problems.

AB - This chapter introduces direct neural dynamic programming (direct NDP), which belongs to the class of heuristic dynamic programming algorithms discussed in Chapters 3, 4, and 19. However, direct NDP is a model-independent approach to action-dependent heuristic dynamic programming. It is, therefore, an on-line learning control paradigm. This chapter contains a comparison study using other well-known algorithms to help readers gain quantitative insight on several ADP algorithms. It also contains results of direct NDP controlling a triple-link inverted pendulum using many continuous state variables and a continuous control, and direct NDP in a wireless network call admission control application. Furthermore, in Chapter 21 direct NDP is demonstrated on an industrial scale Apache helicopter model for stabilization, tracking control, and reconfiguration after component failure. Preliminary results indicate that direct NDP has the potential to address large-scale problems.

KW - Adaptation model

KW - Approximation methods

KW - Artificial neural networks

KW - Dynamic programming

KW - Learning

KW - Learning systems

KW - Markov processes

UR - http://www.scopus.com/inward/record.url?scp=84893246005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893246005&partnerID=8YFLogxK

U2 - 10.1109/9780470544785.ch5

DO - 10.1109/9780470544785.ch5

M3 - Chapter

AN - SCOPUS:84893246005

SN - 047166054X

SN - 9780471660545

SP - 125

EP - 151

BT - Handbook of Learning and Approximate Dynamic Programming

PB - John Wiley and Sons Inc.

ER -