TY - GEN
T1 - A performance gradient perspective on approximate dynamic programming and its application to partially observable markov decision processes
AU - Dankert, James
AU - Yang, Lei
AU - Si, Jennie
PY - 2006
Y1 - 2006
N2 - This paper shows an approach to integrating common approximate dynamic programming (ADP) algorithms into a theoretical framework to address both analytical characteristics and algorithmic features. Several important insights are gained from this analysis, including new approaches to the creation of algorithms. Built on this paradigm, ADP learning algorithms are further developed to address a broader class of problems: optimization with partial observability. This framework is based on an average cost formulation which makes use of the concepts of differential costs and performance gradients to describe learning and optimization algorithms. Numerical simulations are conducted including a queueing problem and a maze problem to illustrate and verify features of the proposed algorithms. Pathways for applying this analysis to adaptive critics are also shown.
AB - This paper shows an approach to integrating common approximate dynamic programming (ADP) algorithms into a theoretical framework to address both analytical characteristics and algorithmic features. Several important insights are gained from this analysis, including new approaches to the creation of algorithms. Built on this paradigm, ADP learning algorithms are further developed to address a broader class of problems: optimization with partial observability. This framework is based on an average cost formulation which makes use of the concepts of differential costs and performance gradients to describe learning and optimization algorithms. Numerical simulations are conducted including a queueing problem and a maze problem to illustrate and verify features of the proposed algorithms. Pathways for applying this analysis to adaptive critics are also shown.
UR - http://www.scopus.com/inward/record.url?scp=61849156138&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=61849156138&partnerID=8YFLogxK
U2 - 10.1109/ISIC.2006.285595
DO - 10.1109/ISIC.2006.285595
M3 - Conference contribution
AN - SCOPUS:61849156138
SN - 0780397983
SN - 9780780397989
T3 - IEEE International Symposium on Intelligent Control - Proceedings
SP - 458
EP - 463
BT - Proceedings of the 2006 IEEE International Symposium on Intelligent Control, ISIC
T2 - Joint 2006 IEEE Conference on Control Applications (CCA), Computer-Aided Control Systems Design Symposium (CACSD) and International Symposium on Intelligent Control (ISIC)
Y2 - 4 October 2006 through 6 October 2006
ER -