Robust optimality for discounted infinite-horizon Markov decision processes with uncertain transition matrices

Baohua Li, Jennie Si

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

We study finite-state, finite-action, discounted infinite-horizon Markov decision processes with uncertain transition matrices in the deterministic policy space. The transition matrices are classified as either independent or correlated. A generalized robust optimality criterion which can be degenerated to some popular optimality criteria is proposed, under which an optimal or near-optimal policy exists for any uncertain transition matrix. Theorems are developed to guarantee a stationary policy being optimal or near-optimal in the deterministic policy space.

Original languageEnglish (US)
Pages (from-to)2112-2116
Number of pages5
JournalIEEE Transactions on Automatic Control
Volume53
Issue number9
DOIs
StatePublished - 2008

Keywords

  • Markov decision processes
  • Robust optimality criterion
  • Uncertain transition matrix

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this