Eventually-stationary policies for markov decision models with non-constant discounting

Yair Carmon; Adam Shwartz

doi:10.4108/ICST.VALUETOOLS2008.4392

Eventually-stationary policies for markov decision models with non-constant discounting

Yair Carmon, Adam Shwartz^*

^*Corresponding author for this work

Technion-Israel Institute of Technology

Research output: Contribution to conference › Paper › peer-review

1 Scopus citations

Abstract

We investigate the existance of simple policies in finite discounted cost Markov Decision Processes, when the discount factor is not constant. We introduce a class called "exponentially representable" discount functions. Within this class we prove existence of optimal policies which are eventually stationary-from some time N onward, and provide an algorithm for their computation. Outside this class, optimal policies with this structure in general do not exist.

Original language	English
DOIs	https://doi.org/10.4108/ICST.VALUETOOLS2008.4392
State	Published - 2008
Externally published	Yes
Event	3rd International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2008 - Athens, Greece Duration: 20 Oct 2008 → 24 Oct 2008

Conference

Conference	3rd International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2008
Country/Territory	Greece
City	Athens
Period	20/10/08 → 24/10/08

Keywords

Discounted cost
General discounting function
Hyperbolic discounting
Markov decision processes
Mixed discounting

Access to Document

10.4108/ICST.VALUETOOLS2008.4392

Cite this

@conference{84484c347e0f44919c839e67b2990ce6,

title = "Eventually-stationary policies for markov decision models with non-constant discounting",

abstract = "We investigate the existance of simple policies in finite discounted cost Markov Decision Processes, when the discount factor is not constant. We introduce a class called {"}exponentially representable{"} discount functions. Within this class we prove existence of optimal policies which are eventually stationary-from some time N onward, and provide an algorithm for their computation. Outside this class, optimal policies with this structure in general do not exist.",

keywords = "Discounted cost, General discounting function, Hyperbolic discounting, Markov decision processes, Mixed discounting",

author = "Yair Carmon and Adam Shwartz",

year = "2008",

doi = "10.4108/ICST.VALUETOOLS2008.4392",

language = "אנגלית",

note = "3rd International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2008 ; Conference date: 20-10-2008 Through 24-10-2008",

}

TY - CONF

T1 - Eventually-stationary policies for markov decision models with non-constant discounting

AU - Carmon, Yair

AU - Shwartz, Adam

PY - 2008

Y1 - 2008

N2 - We investigate the existance of simple policies in finite discounted cost Markov Decision Processes, when the discount factor is not constant. We introduce a class called "exponentially representable" discount functions. Within this class we prove existence of optimal policies which are eventually stationary-from some time N onward, and provide an algorithm for their computation. Outside this class, optimal policies with this structure in general do not exist.

AB - We investigate the existance of simple policies in finite discounted cost Markov Decision Processes, when the discount factor is not constant. We introduce a class called "exponentially representable" discount functions. Within this class we prove existence of optimal policies which are eventually stationary-from some time N onward, and provide an algorithm for their computation. Outside this class, optimal policies with this structure in general do not exist.

KW - Discounted cost

KW - General discounting function

KW - Hyperbolic discounting

KW - Markov decision processes

KW - Mixed discounting

UR - http://www.scopus.com/inward/record.url?scp=84896954185&partnerID=8YFLogxK

U2 - 10.4108/ICST.VALUETOOLS2008.4392

DO - 10.4108/ICST.VALUETOOLS2008.4392

M3 - ???researchoutput.researchoutputtypes.contributiontoconference.paper???

AN - SCOPUS:84896954185

T2 - 3rd International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2008

Y2 - 20 October 2008 through 24 October 2008

ER -

Eventually-stationary policies for markov decision models with non-constant discounting

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this