## Abstract

We study the properties of finite ergodic Markov Chains whose transition probability matrix P is singular. The results establish bounds on the convergence time of P^{m} to a matrix where all the rows are equal to the stationary distribution of P. The results suggest a simple rule for identifying the singular matrices which do not have a finite convergence time. We next study finite convergence to the stationary distribution independent of the initial distribution. The results establish the connection between the convergence time and the order of the minimal polynomial of the transition probability matrix. A queuing problem and a maintenance Markovian decision problem which possess the property of rapid convergence are presented.

Original language | English |
---|---|

Pages (from-to) | 247-253 |

Number of pages | 7 |

Journal | Stochastic Processes and their Applications |

Volume | 7 |

Issue number | 3 |

DOIs | |

State | Published - Aug 1978 |

Externally published | Yes |

## Keywords

- Markov chains
- Markov decision problem
- accessibility
- convergence time
- eigenvalues
- leading vectors
- minimal polynomial
- null space