Dueling Convex Optimization with General Preferences

We address the problem of convex optimization with dueling feedback, where the goal is to minimize a convex function given a weaker form of dueling feedback. Each query consists of two points and the dueling feedback returns a (noisy) single-bit binary comparison of the function values of the two queried points. The translation of the function values to the single comparison bit is through a transfer function. This problem has been addressed previously for some restricted classes of transfer functions, but here we consider a very general transfer function class which includes all functions that admit a series expansion about the origin. Our main contribution is an efficient algorithm with convergence rate of O(ϵ^−4p ) for smooth convex functions, and an optimal rate of^Õ(ϵ−2p ) when the objective is both smooth and strongly convex, where p is the minimal degree (with a non-zero coefficient) in the transfer’s series expansion about the origin.