A K-step look-ahead analysis of Value Iteration algorithms for Markov decision processes

Meir Herzberg*, Uri Yechiali

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


We introduce and analyze a general look-ahead approach for Value Iteration Algorithms used in solving both discounted and undiscounted Markov decision processes. This approach, based on the value-oriented concept interwoven with multiple adaptive relaxation factors, leads to accelerating procedures which perform better than the separate use of either the concept of value oriented or of relaxation. Evaluation and computational considerations of this method are discussed, practical guidelines for implementation are suggested and the suitability of enhancing the method by incorporating Phase 0, Action Elimination procedures and Parallel Processing is indicated. The method was successfully applied to several real problems. We present some numerical results which support the superiority of the developed approach, particularly for undiscounted cases, over other Value Iteration variants.

Original languageEnglish
Pages (from-to)622-636
Number of pages15
JournalEuropean Journal of Operational Research
Issue number3
StatePublished - 8 Feb 1996


  • Adaptive relaxation factor
  • Look-ahead analysis
  • Markov processes
  • Modified policy iteration
  • Value iteration


Dive into the research topics of 'A K-step look-ahead analysis of Value Iteration algorithms for Markov decision processes'. Together they form a unique fingerprint.

Cite this