Markov decision processes with exponentially representable discounting

Yair Carmon, Adam Shwartz*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


We generalize the geometric discount of finite discounted cost Markov Decision Processes to "exponentially representable"discount functions, prove existence of optimal policies which are stationary from some time N onward, and provide an algorithm for their computation. Outside this class, optimal "N-stationary" policies in general do not exist.

Original languageEnglish
Pages (from-to)51-55
Number of pages5
JournalOperations Research Letters
Issue number1
StatePublished - Jan 2009
Externally publishedYes


  • Discounted cost
  • General discounting function
  • Hyperbolic discounting
  • Markov decision processes
  • Mixed discounting


Dive into the research topics of 'Markov decision processes with exponentially representable discounting'. Together they form a unique fingerprint.

Cite this