TY - JOUR
T1 - Controllability and decentralized stabilization of the Kolmogorov forward equation for Markov chains
AU - Elamvazhuthi, Karthik
AU - Biswal, Shiba
AU - Berman, Spring
N1 - Funding Information:
This work was supported by ONR Young Investigator Award, USA N00014-16- 1-2605 and by the Arizona State University, USA Global Security Initiative. The material in this paper was partially presented at the 56th IEEE Conference on Decision and Control, December 12–15, 2017, Melbourne, Australia. This paper was recommended for publication in revised form by Associate Editor Bert Tanner under the direction of Editor Christos G. Cassandras.
Publisher Copyright:
© 2020
PY - 2021/2
Y1 - 2021/2
N2 - In this paper, we provide several results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous-time Markov chain (CTMC) evolving on a finite state space, with the transition rates defined as the control parameters. First, we show that any target probability distribution can be reached asymptotically using time-varying control parameters. Second, we characterize all stationary distributions that are stabilizable using time-independent control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize stationary distributions while satisfying the additional constraint that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of robots modeled as a CTMC to a target state distribution with no state-switching at equilibrium. In addition to these results, we prove a sufficient condition under which the classical rank conditions for controllability can be generalized to forward equations with non-negativity constraints on the control inputs. We apply this result to prove local controllability of a forward equation in which only a small subset of the transition rates are the control inputs. Lastly, we extend our feedback stabilization results to stationary distributions that have a strongly connected support.
AB - In this paper, we provide several results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous-time Markov chain (CTMC) evolving on a finite state space, with the transition rates defined as the control parameters. First, we show that any target probability distribution can be reached asymptotically using time-varying control parameters. Second, we characterize all stationary distributions that are stabilizable using time-independent control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize stationary distributions while satisfying the additional constraint that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of robots modeled as a CTMC to a target state distribution with no state-switching at equilibrium. In addition to these results, we prove a sufficient condition under which the classical rank conditions for controllability can be generalized to forward equations with non-negativity constraints on the control inputs. We apply this result to prove local controllability of a forward equation in which only a small subset of the transition rates are the control inputs. Lastly, we extend our feedback stabilization results to stationary distributions that have a strongly connected support.
KW - Autonomous mobile robots
KW - Bilinear control systems
KW - Continuous-time Markov chains
KW - Controllability
KW - Swarm robotics
UR - http://www.scopus.com/inward/record.url?scp=85096864936&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096864936&partnerID=8YFLogxK
U2 - 10.1016/j.automatica.2020.109351
DO - 10.1016/j.automatica.2020.109351
M3 - Article
AN - SCOPUS:85096864936
SN - 0005-1098
VL - 124
JO - Automatica
JF - Automatica
M1 - 109351
ER -