Random-Sampling Monte-Carlo Tree Search Methods for Cost Approximation in Long-Horizon Optimal Control

Shankarachary Ragi, Hans D. Mittelmann

Research output: Contribution to journalArticlepeer-review

Abstract

We develop Monte-Carlo based heuristic approaches to approximate the objective function in long horizon optimal control problems. In these approaches, to approximate the expectation operator in the objective function, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the average (or weighted average) of the costs along all the trajectories. We call these methods random sampling - multipath hypothesis propagation or RS-MHP. These methods (or variants) exist in the literature; however, the literature lacks results on how well these approximation strategies converge. This letter fills this knowledge gap to a certain extent. We derive stochastic convergence results for the cost approximation error from the RS-MHP methods and discuss their convergence (in probability) as the sample size increases. We consider two case studies to demonstrate the effectiveness of our methods - a) linear quadratic control problem; b) unmanned aerial vehicle path optimization problem.

Original languageEnglish (US)
Article number9289842
Pages (from-to)1759-1764
Number of pages6
JournalIEEE Control Systems Letters
Volume5
Issue number5
DOIs
StatePublished - Nov 2021

Keywords

  • Markov processes
  • Optimal control
  • discrete event systems
  • optimization

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Control and Optimization

Fingerprint Dive into the research topics of 'Random-Sampling Monte-Carlo Tree Search Methods for Cost Approximation in Long-Horizon Optimal Control'. Together they form a unique fingerprint.

Cite this