Stable optimal control and semicontractive dynamic programming

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

We consider discrete-time infinite horizon deterministic optimal control problems with nonnegative cost per stage, and a destination that is cost free and absorbing. The classical linear-quadratic regulator problem is a special case. Our assumptions are very general, and allow the possibility that the optimal policy May not be stabilizing the system, e.g., May not reach the destination either asymptotically or in a finite number of steps. We introduce a new unifying notion of stable feedback policy, based on perturbation of the cost per stage, which in addition to implying convergence of the generated States to the destination, quantifies the speed of convergence. We consider the properties of two distinct cost functions: J, the overall optimal, and Ĵ, the restricted optimal over just the stable policies. Different classes of stable policies (with different speeds of convergence) May yield different values of Ĵ. We show that for any class of stable policies, Ĵ is a solution of Bellman’s equation, and we characterize the smallest and the largest solutions: they are J, and J+, the restricted optimal cost function over the class of (finitely) terminating policies. We also characterize the regions of convergence of various modified versions of value and policy iteration algorithms, as substitutes for the standard algorithms, which May not work in general.

Original languageEnglish (US)
Pages (from-to)231-252
Number of pages22
JournalSIAM Journal on Control and Optimization
Volume56
Issue number1
DOIs
StatePublished - 2018
Externally publishedYes

Keywords

  • Discrete-time optimal control
  • Dynamic programming
  • Policy iteration
  • Shortest path
  • Stable policy
  • Value iteration

ASJC Scopus subject areas

  • Control and Optimization
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Stable optimal control and semicontractive dynamic programming'. Together they form a unique fingerprint.

Cite this