Controllability and decentralized stabilization of the Kolmogorov forward equation for Markov chains

Karthik Elamvazhuthi, Shiba Biswal, Spring Berman

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we provide several results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous-time Markov chain (CTMC) evolving on a finite state space, with the transition rates defined as the control parameters. First, we show that any target probability distribution can be reached asymptotically using time-varying control parameters. Second, we characterize all stationary distributions that are stabilizable using time-independent control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize stationary distributions while satisfying the additional constraint that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of robots modeled as a CTMC to a target state distribution with no state-switching at equilibrium. In addition to these results, we prove a sufficient condition under which the classical rank conditions for controllability can be generalized to forward equations with non-negativity constraints on the control inputs. We apply this result to prove local controllability of a forward equation in which only a small subset of the transition rates are the control inputs. Lastly, we extend our feedback stabilization results to stationary distributions that have a strongly connected support.

Original languageEnglish (US)
Article number109351
JournalAutomatica
Volume124
DOIs
StatePublished - Feb 2021

Keywords

  • Autonomous mobile robots
  • Bilinear control systems
  • Continuous-time Markov chains
  • Controllability
  • Swarm robotics

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Controllability and decentralized stabilization of the Kolmogorov forward equation for Markov chains'. Together they form a unique fingerprint.

Cite this