Decentralized State-Dependent Markov Chain Synthesis with an Application to Swarm Guidance

Uzun S., Üre N. K., Acikmese B.

IEEE Transactions on Automatic Control, 2024 (SCI-Expanded) identifier

  • Publication Type: Article / Article
  • Publication Date: 2024
  • Doi Number: 10.1109/tac.2024.3353673
  • Journal Name: IEEE Transactions on Automatic Control
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, INSPEC, MathSciNet, Metadex, zbMATH, Civil Engineering Abstracts
  • Keywords: Consensus protocol, Consensus Protocol, Convergence, Decentralized Control, Heuristic algorithms, Markov Chains, Markov processes, Network topology, Probabilistic logic, Probabilistic Swarm Guidance, Topology
  • Istanbul Technical University Affiliated: Yes


This paper introduces a decentralized state-dependent Markov chain synthesis (DSMC) algorithm for finite-state Markov chains. We present a state-dependent consensus protocol that achieves exponential convergence under mild technical conditions, without relying on any connectivity assumptions regarding the dynamic network topology. Utilizing the proposed consensus protocol, we develop the DSMC algorithm, updating the Markov matrix based on the current state while ensuring the convergence conditions of the consensus protocol. This result establishes the desired steady-state distribution for the resulting Markov chain, ensuring exponential convergence from all initial distributions while adhering to transition constraints and minimizing state transitions. The DSMC's performance is demonstrated through a probabilistic swarm guidance example, which interprets the spatial distribution of a swarm comprising a large number of mobile agents as a probability distribution and utilizes the Markov chain to compute transition probabilities between states. Simulation results demonstrate faster convergence for the DSMC based algorithm when compared to the previous Markov chain based swarm guidance algorithms.