Private Online Prediction from Experts: Separations and Faster Rates

Hilal Asi, Vitaly Feldman, Tomer Koren, Kunal Talwar

Research output: Contribution to journalConference articlepeer-review

Abstract

Online prediction from experts is a fundamental problem in machine learning and several works have studied this problem under privacy constraints. We propose and analyze new algorithms for this problem that improve over the regret bounds of the best existing algorithms for non-adaptive adversaries. For approximate differential privacy, our algorithms achieve regret bounds of Oe(√T log d + log d/ε) for the stochastic setting and Oe(√T log d + T1/3 log d/ε) for oblivious adversaries (where d is the number of experts). For pure DP, our algorithms are the first to obtain sub-linear regret for oblivious adversaries in the high-dimensional regime d ≥ T. Moreover, we prove new lower bounds for adaptive adversaries. Our results imply that unlike the non-private setting, there is a strong separation between the optimal regret for adaptive and non-adaptive adversaries for this problem. Our lower bounds also show a separation between pure and approximate differential privacy for adaptive adversaries where the latter is necessary to achieve the non-private O(T) regret.

Original languageEnglish
Pages (from-to)674-699
Number of pages26
JournalProceedings of Machine Learning Research
Volume195
StatePublished - 2023
Event36th Annual Conference on Learning Theory, COLT 2023 - Bangalore, India
Duration: 12 Jul 202315 Jul 2023

Fingerprint

Dive into the research topics of 'Private Online Prediction from Experts: Separations and Faster Rates'. Together they form a unique fingerprint.

Cite this