TY - JOUR

T1 - Acceleration with a ball optimization oracle

AU - Carmon, Yair

AU - Jambulapati, Arun

AU - Jiang, Qijia

AU - Jin, Yujia

AU - Lee, Yin Tat

AU - Sidford, Aaron

AU - Tian, Kevin

N1 - Publisher Copyright:
© 2020 Neural information processing systems foundation. All rights reserved.

PY - 2020

Y1 - 2020

N2 - Consider an oracle which takes a point x and returns the minimizer of a convex function f in an l2 ball of radius r around x. It is straightforward to show that roughly r-1 log 1e calls to the oracle suffice to find an e-approximate minimizer of f in an l2 unit ball. Perhaps surprisingly, this is not optimal: we design an accelerated algorithm which attains an e-approximate minimizer with roughly r-2/3 log 1e oracle queries, and give a matching lower bound. Further, we implement ball optimization oracles for functions with locally stable Hessians using a variant of Newton’s method and, in certain cases, stochastic first-order methods. The resulting algorithm applies to a number of problems of practical and theoretical import, improving upon previous results for logistic and l8 regression and achieving guarantees comparable to the state-of-the-art for lp regression.

AB - Consider an oracle which takes a point x and returns the minimizer of a convex function f in an l2 ball of radius r around x. It is straightforward to show that roughly r-1 log 1e calls to the oracle suffice to find an e-approximate minimizer of f in an l2 unit ball. Perhaps surprisingly, this is not optimal: we design an accelerated algorithm which attains an e-approximate minimizer with roughly r-2/3 log 1e oracle queries, and give a matching lower bound. Further, we implement ball optimization oracles for functions with locally stable Hessians using a variant of Newton’s method and, in certain cases, stochastic first-order methods. The resulting algorithm applies to a number of problems of practical and theoretical import, improving upon previous results for logistic and l8 regression and achieving guarantees comparable to the state-of-the-art for lp regression.

UR - http://www.scopus.com/inward/record.url?scp=85107884743&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???

AN - SCOPUS:85107884743

SN - 1049-5258

VL - 2020-December

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - 34th Conference on Neural Information Processing Systems, NeurIPS 2020

Y2 - 6 December 2020 through 12 December 2020

ER -