Faster Convergence with Multiway Preferences

Aadirupa Saha, Vitaly Feldman, Tomer Koren, Yishay Mansour

Research output: Contribution to journalConference articlepeer-review

Abstract

We address the problem of convex optimization with preference feedback, where the goal is to minimize a convex function given a weaker form of comparison queries. Each query consists of two points and the dueling feedback returns a (noisy) single-bit binary comparison of the function values of the two queried points. Here we consider the sign-function-based comparison feedback model and analyze the convergence rates with batched and multiway (argmin of a set queried points) comparisons. Our main goal is to understand the improved convergence rates owing to parallelization in sign-feedback-based optimization problems. Our work is the first to study the problem of convex optimization with multiway preferences and analyze the optimal convergence rates. Our first contribution lies in designing efficient algorithms with a convergence rate of Oe(min{dm,d}ϵ) for mbatched preference feedback where the learner can query m-pairs in parallel. We next study a m-multiway comparison (‘battling’) feedback, where the learner can get to see the argmin feedback of m-subset of queried points and show a convergence rate of Oe(min{logdm,d}ϵ). We show further improved convergence rates with an additional assumption of strong convexity. Finally, we also study the convergence lower bounds for batched preferences and multiway feedback optimization showing the optimality of our convergence rates w.r.t. m.

Original languageEnglish
Pages (from-to)433-441
Number of pages9
JournalProceedings of Machine Learning Research
Volume238
StatePublished - 2024
Event27th International Conference on Artificial Intelligence and Statistics, AISTATS 2024 - Valencia, Spain
Duration: 2 May 20244 May 2024

Funding

FundersFunder number
Blavatnik Family Foundation
Aegis Foundation
Tel Aviv University
European Research Council
Horizon 2020 Framework Programme882396, 101078075
Horizon 2020 Framework Programme
Israel Science Foundation2549/19, 2250/22
Israel Science Foundation

    Fingerprint

    Dive into the research topics of 'Faster Convergence with Multiway Preferences'. Together they form a unique fingerprint.

    Cite this