Thompson Sampling for Adversarial Bit Prediction

Research output: Contribution to journalConference articlepeer-review

Abstract

We study the Thompson sampling algorithm in an adversarial setting, specifically, for adversarial bit prediction. We characterize the bit sequences with the smallest and largest expected regret. Among sequences of length T with k < T2 zeros, the sequences of largest regret consist of alternating zeros and ones followed by the remaining ones, and the sequence of smallest regret consists of ones followed_by zeros. We also bound the regret of those sequences, the worst case sequences have regret O(T) and the best case sequence have regret O(1). We extend our results to a model where false positive and false negative errors have different weights. We characterize the sequences with largest expected regret in this generalized setting, and derive their regret bounds. We also show that there are sequences with O(1) regret.

Original languageEnglish
Pages (from-to)518-553
Number of pages36
JournalProceedings of Machine Learning Research
Volume117
StatePublished - 2020
Event31st International Conference on Algorithmic Learning Theory, ALT 2020 - San Diego, United States
Duration: 8 Feb 202011 Feb 2020

Keywords

  • Adversarial setting
  • Bit prediction
  • Multi-armed bandits
  • Regret
  • Thompson sampling

Fingerprint

Dive into the research topics of 'Thompson Sampling for Adversarial Bit Prediction'. Together they form a unique fingerprint.

Cite this