Mean-field controllability and decentralized stabilization of Markov chains

Karthik Elamvazhuthi, Matthias Kawski, Shiba Biswal, Vaibhav Deshmukh, Spring Berman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.

Original languageEnglish (US)
Title of host publication2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3131-3137
Number of pages7
Volume2018-January
ISBN (Electronic)9781509028733
DOIs
StatePublished - Jan 18 2018
Event56th IEEE Annual Conference on Decision and Control, CDC 2017 - Melbourne, Australia
Duration: Dec 12 2017Dec 15 2017

Other

Other56th IEEE Annual Conference on Decision and Control, CDC 2017
CountryAustralia
CityMelbourne
Period12/12/1712/15/17

Fingerprint

Feedback Law
Continuous-time Markov Chain
Controllability
Control Parameter
Mean Field
Markov processes
Decentralized
Markov chain
Stabilization
Strictly positive
Stationary Distribution
Feedback
Time-varying Parameters
Target
Polynomial
Stabilizability
Mean-field Model
Equilibrium Distribution
Stochastic Simulation
Local Time

ASJC Scopus subject areas

  • Decision Sciences (miscellaneous)
  • Industrial and Manufacturing Engineering
  • Control and Optimization

Cite this

Elamvazhuthi, K., Kawski, M., Biswal, S., Deshmukh, V., & Berman, S. (2018). Mean-field controllability and decentralized stabilization of Markov chains. In 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017 (Vol. 2018-January, pp. 3131-3137). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2017.8264117

Mean-field controllability and decentralized stabilization of Markov chains. / Elamvazhuthi, Karthik; Kawski, Matthias; Biswal, Shiba; Deshmukh, Vaibhav; Berman, Spring.

2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 3131-3137.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Elamvazhuthi, K, Kawski, M, Biswal, S, Deshmukh, V & Berman, S 2018, Mean-field controllability and decentralized stabilization of Markov chains. in 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 3131-3137, 56th IEEE Annual Conference on Decision and Control, CDC 2017, Melbourne, Australia, 12/12/17. https://doi.org/10.1109/CDC.2017.8264117
Elamvazhuthi K, Kawski M, Biswal S, Deshmukh V, Berman S. Mean-field controllability and decentralized stabilization of Markov chains. In 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 3131-3137 https://doi.org/10.1109/CDC.2017.8264117
Elamvazhuthi, Karthik ; Kawski, Matthias ; Biswal, Shiba ; Deshmukh, Vaibhav ; Berman, Spring. / Mean-field controllability and decentralized stabilization of Markov chains. 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 3131-3137
@inproceedings{585cf980af854997b590d65e75c16837,
title = "Mean-field controllability and decentralized stabilization of Markov chains",
abstract = "In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.",
author = "Karthik Elamvazhuthi and Matthias Kawski and Shiba Biswal and Vaibhav Deshmukh and Spring Berman",
year = "2018",
month = "1",
day = "18",
doi = "10.1109/CDC.2017.8264117",
language = "English (US)",
volume = "2018-January",
pages = "3131--3137",
booktitle = "2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Mean-field controllability and decentralized stabilization of Markov chains

AU - Elamvazhuthi, Karthik

AU - Kawski, Matthias

AU - Biswal, Shiba

AU - Deshmukh, Vaibhav

AU - Berman, Spring

PY - 2018/1/18

Y1 - 2018/1/18

N2 - In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.

AB - In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.

UR - http://www.scopus.com/inward/record.url?scp=85046136590&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046136590&partnerID=8YFLogxK

U2 - 10.1109/CDC.2017.8264117

DO - 10.1109/CDC.2017.8264117

M3 - Conference contribution

VL - 2018-January

SP - 3131

EP - 3137

BT - 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -