Learning rates for Q-learning

Eyal Even-Dar, Yishay Mansour

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In this paper we derive convergence rates for Q-learning. We show an interesting relationship between the convergence rate and the learning rate used in the Q-learning. For a polynomial learning rate, one which is 1/tω at time t where ω ∈ (1/2, 1), we show that that the convergence rate is polynomial in 1/(1 − γ), where γ is the discount factor. In contrast we show that for a linear learning rate, one which is 1/t at time t, the convergence rate has an exponential dependence on 1/(1 − γ). In addition we show a simple example that proves that this exponential behavior is inherent for a linear learning rate.

Original languageEnglish
Title of host publicationComputational Learning Theory - 14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001, Proceedings
EditorsDavid Helmbold, Bob Williamson
PublisherSpringer Verlag
Pages589-604
Number of pages16
ISBN (Print)9783540423430
DOIs
StatePublished - 2001
Event14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001 - Amsterdam, Netherlands
Duration: 16 Jul 200119 Jul 2001

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2111
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001
Country/TerritoryNetherlands
CityAmsterdam
Period16/07/0119/07/01

Fingerprint

Dive into the research topics of 'Learning rates for Q-learning'. Together they form a unique fingerprint.

Cite this