Discounted deterministic Markov decision processes and discounted all-pairs shortest paths

Omid Madani; Mikkel Thorup; Uri Zwick

doi:10.1137/1.9781611973068.104

Discounted deterministic Markov decision processes and discounted all-pairs shortest paths

Omid Madani^*, Mikkel Thorup, Uri Zwick

^*Corresponding author for this work

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We present two new algorithms for finding optimal strategies for discounted, infinite-horizon, Deterministic Markov Decision Processes (DMDP). The first one is an adaptation of an algorithm of Young, Tarjan and Orlin for finding minimum mean weight cycles. It runs in O(mn + n² log n) time, where n is the number of vertices (or states) and m is the number of edges (or actions). The second one is an adaptation of a classical algorithm of Karp for finding minimum mean weight cycles. It runs in O(mn) time. The first algorithm has a slightly slower worst-case complexity, but is faster than the first algorithm in many situations. Both algorithms improve on a recent O(mn ²)-time algorithm of Andersson and Vorobyov. We also present a randomized Õ(m^1/2n²)-time algorithm for finding Discounted All-Pairs Shortest Paths (DAPSP), improving several previous algorithms.

Original language	English
Title of host publication	Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms
Publisher	Association for Computing Machinery (ACM)
Pages	958-967
Number of pages	10
ISBN (Print)	9780898716801
DOIs	https://doi.org/10.1137/1.9781611973068.104
State	Published - 2009
Event	20th Annual ACM-SIAM Symposium on Discrete Algorithms - New York, NY, United States Duration: 4 Jan 2009 → 6 Jan 2009

Publication series

Name	Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms

Conference

Conference	20th Annual ACM-SIAM Symposium on Discrete Algorithms
Country/Territory	United States
City	New York, NY
Period	4/01/09 → 6/01/09

Access to Document

10.1137/1.9781611973068.104

Cite this

Madani, O., Thorup, M., & Zwick, U. (2009). Discounted deterministic Markov decision processes and discounted all-pairs shortest paths. In Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (pp. 958-967). (Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms). Association for Computing Machinery (ACM). https://doi.org/10.1137/1.9781611973068.104

@inproceedings{c9409aacc4f34cf5bffa9a8c8ff0cd6f,

title = "Discounted deterministic Markov decision processes and discounted all-pairs shortest paths",

abstract = "We present two new algorithms for finding optimal strategies for discounted, infinite-horizon, Deterministic Markov Decision Processes (DMDP). The first one is an adaptation of an algorithm of Young, Tarjan and Orlin for finding minimum mean weight cycles. It runs in O(mn + n2 log n) time, where n is the number of vertices (or states) and m is the number of edges (or actions). The second one is an adaptation of a classical algorithm of Karp for finding minimum mean weight cycles. It runs in O(mn) time. The first algorithm has a slightly slower worst-case complexity, but is faster than the first algorithm in many situations. Both algorithms improve on a recent O(mn 2)-time algorithm of Andersson and Vorobyov. We also present a randomized {\~O}(m1/2n2)-time algorithm for finding Discounted All-Pairs Shortest Paths (DAPSP), improving several previous algorithms.",

author = "Omid Madani and Mikkel Thorup and Uri Zwick",

year = "2009",

doi = "10.1137/1.9781611973068.104",

language = "אנגלית",

isbn = "9780898716801",

series = "Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms",

publisher = "Association for Computing Machinery (ACM)",

pages = "958--967",

booktitle = "Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms",

address = "ארצות הברית",

note = "20th Annual ACM-SIAM Symposium on Discrete Algorithms ; Conference date: 04-01-2009 Through 06-01-2009",

}

Madani, O, Thorup, M & Zwick, U 2009, Discounted deterministic Markov decision processes and discounted all-pairs shortest paths. in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms. Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, Association for Computing Machinery (ACM), pp. 958-967, 20th Annual ACM-SIAM Symposium on Discrete Algorithms, New York, NY, United States, 4/01/09. https://doi.org/10.1137/1.9781611973068.104

Discounted deterministic Markov decision processes and discounted all-pairs shortest paths. / Madani, Omid; Thorup, Mikkel; Zwick, Uri.
Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms. Association for Computing Machinery (ACM), 2009. p. 958-967 (Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Discounted deterministic Markov decision processes and discounted all-pairs shortest paths

AU - Madani, Omid

AU - Thorup, Mikkel

AU - Zwick, Uri

PY - 2009

Y1 - 2009

N2 - We present two new algorithms for finding optimal strategies for discounted, infinite-horizon, Deterministic Markov Decision Processes (DMDP). The first one is an adaptation of an algorithm of Young, Tarjan and Orlin for finding minimum mean weight cycles. It runs in O(mn + n2 log n) time, where n is the number of vertices (or states) and m is the number of edges (or actions). The second one is an adaptation of a classical algorithm of Karp for finding minimum mean weight cycles. It runs in O(mn) time. The first algorithm has a slightly slower worst-case complexity, but is faster than the first algorithm in many situations. Both algorithms improve on a recent O(mn 2)-time algorithm of Andersson and Vorobyov. We also present a randomized Õ(m1/2n2)-time algorithm for finding Discounted All-Pairs Shortest Paths (DAPSP), improving several previous algorithms.

AB - We present two new algorithms for finding optimal strategies for discounted, infinite-horizon, Deterministic Markov Decision Processes (DMDP). The first one is an adaptation of an algorithm of Young, Tarjan and Orlin for finding minimum mean weight cycles. It runs in O(mn + n2 log n) time, where n is the number of vertices (or states) and m is the number of edges (or actions). The second one is an adaptation of a classical algorithm of Karp for finding minimum mean weight cycles. It runs in O(mn) time. The first algorithm has a slightly slower worst-case complexity, but is faster than the first algorithm in many situations. Both algorithms improve on a recent O(mn 2)-time algorithm of Andersson and Vorobyov. We also present a randomized Õ(m1/2n2)-time algorithm for finding Discounted All-Pairs Shortest Paths (DAPSP), improving several previous algorithms.

UR - http://www.scopus.com/inward/record.url?scp=70349115580&partnerID=8YFLogxK

U2 - 10.1137/1.9781611973068.104

DO - 10.1137/1.9781611973068.104

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:70349115580

SN - 9780898716801

T3 - Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms

SP - 958

EP - 967

BT - Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms

PB - Association for Computing Machinery (ACM)

T2 - 20th Annual ACM-SIAM Symposium on Discrete Algorithms

Y2 - 4 January 2009 through 6 January 2009

ER -

Discounted deterministic Markov decision processes and discounted all-pairs shortest paths

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this