On near optimality of the set of finite-state controllers for average cost POMDP

Research output: Contribution to journalArticlepeer-review

13 Scopus citations

Abstract

We consider the average cost problem for partially observable Markov decision processes (POMDP) with finite state, observation, and control spaces. We prove that there exists an e-optimal finite-state controller (FSC) functionally independent of initial distributions for any ε > 0, under the assumption that the optimal liminf average cost function of the POMDP is constant. As part of our proof, we establish that if the optimal liminf average cost function is constant, then the optimal limsup average cost function is also constant, and the two are equal. We also discuss the connection between the existence of nearly optimal finite-history controllers and two other important issues for average cost POMDP: the existence of an average cost that is independent of the initial state distribution, and the existence of a bounded solution to the constant average cost optimality equation.

Original languageEnglish (US)
Pages (from-to)1-11
Number of pages11
JournalMathematics of Operations Research
Volume33
Issue number1
DOIs
StatePublished - Feb 2008
Externally publishedYes

Keywords

  • Average cost criterion
  • Finite-state and control models
  • Optimality conditions
  • Partially observable markov decision processes

ASJC Scopus subject areas

  • Mathematics(all)
  • Computer Science Applications
  • Management Science and Operations Research

Fingerprint Dive into the research topics of 'On near optimality of the set of finite-state controllers for average cost POMDP'. Together they form a unique fingerprint.

Cite this