Value iteration method for countable state Markov decision processes

Yossi Aviv, Awi Federgruen

Research output: Contribution to journalArticlepeer-review


This paper deals with Markov decision processes with a countable state space. We demonstrate that a single, relatively simple condition suffices to guarantee that the value-iteration method converges and that an optimal policy can be computed via this method, once the existence of a solution to the average cost optimality equation has been established via any of the many available sets of existence conditions.

Original languageEnglish
Pages (from-to)223-234
Number of pages12
JournalOperations Research Letters
Issue number5
StatePublished - Jun 1999
Externally publishedYes


Dive into the research topics of 'Value iteration method for countable state Markov decision processes'. Together they form a unique fingerprint.

Cite this