TY - GEN
T1 - Bayesian incentive-compatible bandit exploration
AU - Mansour, Yishay
AU - Slivkins, Aleksandrs
AU - Syrgkanis, Vasilis
PY - 2015/6/15
Y1 - 2015/6/15
N2 - Individual decision-makers consume information revealed by the previous decision makers, and produce information that may help in future decision makers. This phenomenon is common in a wide range of sce- narios in the Internet economy, as well as elsewhere, such as medical decisions. Each decision maker when required to select an action, would individually prefer to exploit, select the highest expected reward ac- tion conditional on her information. At the same time, each decision maker would prefer previous decision makers to explore, producing information about the rewards of various actions. A social planner, by means of carefully designed information disclosure, can incentivize the agents to balance the exploration and ex- ploitation, and maximize social welfare. We formulate this problem as a multi-arm bandit problem (and various generalizations thereof) under incentive-compatibility constraints induced by agents' Bayesian priors. We design an incentive-compatible bandit algorithm for the social planner with asymptotically optimal regret. Further, we provide a black- box reduction from an arbitrary multi-arm bandit algorithm to an incentive-compatible one, with only a constant multiplicative increase in regret. This reduction works for very general bandit settings, even ones that incorporate contexts and arbitrary partial feedback.
AB - Individual decision-makers consume information revealed by the previous decision makers, and produce information that may help in future decision makers. This phenomenon is common in a wide range of sce- narios in the Internet economy, as well as elsewhere, such as medical decisions. Each decision maker when required to select an action, would individually prefer to exploit, select the highest expected reward ac- tion conditional on her information. At the same time, each decision maker would prefer previous decision makers to explore, producing information about the rewards of various actions. A social planner, by means of carefully designed information disclosure, can incentivize the agents to balance the exploration and ex- ploitation, and maximize social welfare. We formulate this problem as a multi-arm bandit problem (and various generalizations thereof) under incentive-compatibility constraints induced by agents' Bayesian priors. We design an incentive-compatible bandit algorithm for the social planner with asymptotically optimal regret. Further, we provide a black- box reduction from an arbitrary multi-arm bandit algorithm to an incentive-compatible one, with only a constant multiplicative increase in regret. This reduction works for very general bandit settings, even ones that incorporate contexts and arbitrary partial feedback.
KW - Bayesian incentive-compatibility
KW - Mechanism design
KW - Multi-armed bandits
KW - Regret
UR - http://www.scopus.com/inward/record.url?scp=84962109375&partnerID=8YFLogxK
U2 - 10.1145/2764468.2764508
DO - 10.1145/2764468.2764508
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:84962109375
T3 - EC 2015 - Proceedings of the 2015 ACM Conference on Economics and Computation
SP - 565
EP - 582
BT - EC 2015 - Proceedings of the 2015 ACM Conference on Economics and Computation
PB - Association for Computing Machinery, Inc
T2 - 16th ACM Conference on Economics and Computation, EC 2015
Y2 - 15 June 2015 through 19 June 2015
ER -