## Abstract

We consider stochastic shortest path problems with infinite state and control spaces, a nonnegative cost per stage, and a termination state. We extend the notion of a proper policy, a policy that terminates within a finite expected number of steps, from the context of finite state space to the context of infinite state space. We consider the optimal cost function J∗, and the optimal cost function Ĵ over just the proper policies. We show that J∗ and Ĵ are the smallest and largest solutions of Bellman's equation, respectively, within a suitable class of Lyapounov-like functions. If the cost per stage is bounded, these functions are those that are bounded over the effective domain of Ĵ. The standard value iteration algorithm may be attracted to either J∗ or Ĵ, depending on the initial condition.

Original language | English (US) |
---|---|

Article number | 8309409 |

Pages (from-to) | 3787-3792 |

Number of pages | 6 |

Journal | IEEE Transactions on Automatic Control |

Volume | 63 |

Issue number | 11 |

DOIs | |

State | Published - Nov 2018 |

Externally published | Yes |

## Keywords

- Dynamic programming
- Markov decision processes
- stochastic optimal control
- stochastic shortest paths (SSPs)

## ASJC Scopus subject areas

- Control and Systems Engineering
- Computer Science Applications
- Electrical and Electronic Engineering