TY - GEN
T1 - "Convex until proven guilty"
T2 - 34th International Conference on Machine Learning, ICML 2017
AU - Cannon, Yair
AU - Duchi, John C.
AU - Hinder, Oliver
AU - Sidford, Aaron
N1 - Publisher Copyright:
© 2017 by the author(s).
PY - 2017
Y1 - 2017
N2 - We develop and analyze a variant of Nesterov's accelerated gradient descent (AGD) for minimization of smooth non-convex functions. We prove that one of two cases occurs: either our AGD variant converges quickly, as if the function was convex, or we produce a certificate that the function is "guilty" of being non-convex. This non-convexity certificate allows us to exploit negative curvature and obtain deterministic, dimension-free acceleration of convergence for non-convex functions. For a function /with Lipschitz continuous gradient and Hessian, we compute a point x with ∥Vf(x)∥ ≤ ϵ in O(ϵ-7/4 log(l/ϵ)) gradient and function evaluations. Assuming additionally that the third derivative is Lipschitz, we require only O(ϵ-5/3log(1/ϵ)) evaluations.
AB - We develop and analyze a variant of Nesterov's accelerated gradient descent (AGD) for minimization of smooth non-convex functions. We prove that one of two cases occurs: either our AGD variant converges quickly, as if the function was convex, or we produce a certificate that the function is "guilty" of being non-convex. This non-convexity certificate allows us to exploit negative curvature and obtain deterministic, dimension-free acceleration of convergence for non-convex functions. For a function /with Lipschitz continuous gradient and Hessian, we compute a point x with ∥Vf(x)∥ ≤ ϵ in O(ϵ-7/4 log(l/ϵ)) gradient and function evaluations. Assuming additionally that the third derivative is Lipschitz, we require only O(ϵ-5/3log(1/ϵ)) evaluations.
UR - http://www.scopus.com/inward/record.url?scp=85037544802&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85037544802
T3 - 34th International Conference on Machine Learning, ICML 2017
SP - 1069
EP - 1091
BT - 34th International Conference on Machine Learning, ICML 2017
PB - International Machine Learning Society (IMLS)
Y2 - 6 August 2017 through 11 August 2017
ER -