Random-sampling multipath hypothesis propagation for cost approximation in long-horizon optimal control

Shankarachary Ragi, Hans D. Mittelmann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

In this paper, we develop a Monte-Carlo based heuristic approach to approximate the objective function in long horizon optimal control problems. In this approach, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the weighted average of the costs along all the trajectories. We call these methods random sampling - multipath hypothesis propagation or RS-MHP. These methods (or variants) exist in the literature; however, the literature lacks convergence results for a generic class of nonlinear systems. This paper fills this knowledge gap to a certain extent. We derive convergence results for the cost approximation error from the MHP methods and discuss their convergence (in probability) as the sample size increases. As a case study, we apply RS-MHP to approximate the cost function in a linear quadratic control problem and demonstrate the benefits of our approach against an existing and closely related approximation approach called nominal belief-state optimization.

Original languageEnglish (US)
Title of host publicationCCTA 2020 - 4th IEEE Conference on Control Technology and Applications
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages14-18
Number of pages5
ISBN (Electronic)9781728171401
DOIs
StatePublished - Aug 2020
Event4th IEEE Conference on Control Technology and Applications, CCTA 2020 - Virtual, Montreal, Canada
Duration: Aug 24 2020Aug 26 2020

Publication series

NameCCTA 2020 - 4th IEEE Conference on Control Technology and Applications

Conference

Conference4th IEEE Conference on Control Technology and Applications, CCTA 2020
Country/TerritoryCanada
CityVirtual, Montreal
Period8/24/208/26/20

Keywords

  • Approximate dynamic programming
  • Cost approximation
  • Long horizon optimal control
  • Multipath hypothesis propagation

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Optimization
  • Instrumentation

Fingerprint

Dive into the research topics of 'Random-sampling multipath hypothesis propagation for cost approximation in long-horizon optimal control'. Together they form a unique fingerprint.

Cite this