Complexity Analysis of a Countable-armed Bandit Problem

Anand Kalvit, Assaf Zeevi

Research output: Contribution to journalConference articlepeer-review


We consider a stochastic multi-armed bandit (MAB) problem motivated by “large” action spaces, and endowed with a population of arms containing exactly K arm-types, each characterized by a distinct mean reward. The decision maker is oblivious to the statistical properties of reward distributions as well as the population-level distribution of different arm-types, and is precluded also from observing the type of an arm after play. We study the classical problem of minimizing the expected cumulative regret over a horizon of play n, and propose algorithms that achieve a rate-optimal finite-time instance-dependent regret of O (log n). We also show that the instance-independent (minimax) regret is Õ (√n) when K = 2. While the order of regret and complexity of the problem suggests a great degree of similarity to the classical MAB problem, properties of the performance bounds and salient aspects of algorithm design are quite distinct from the latter, as are the key primitives that determine complexity along with the analysis tools needed to study them.

Original languageEnglish
Pages (from-to)850-890
Number of pages41
JournalProceedings of Machine Learning Research
StatePublished - 2023
Externally publishedYes
Event34th International Conference onAlgorithmic Learning Theory, ALT 2023 - Singapore, Singapore
Duration: 20 Feb 202323 Feb 2023


  • Adaptivity to reservoir-distribution
  • Explore-then-Commit
  • Infinite-armed bandits
  • UCB


Dive into the research topics of 'Complexity Analysis of a Countable-armed Bandit Problem'. Together they form a unique fingerprint.

Cite this