Accelerated methods for nonconvex optimization

Yair Carmon, John C. Duchi, Oliver Hinder, Aaron Sidford

Research output: Contribution to journalArticlepeer-review

122 Scopus citations

Abstract

We present an accelerated gradient method for nonconvex optimization problems with Lipschitz continuous first and second derivatives. In a time O(7/4 log(1/)), the method finds an -stationary point, meaning a point x such that ∇f(x) ≤ . The method improves upon the O( 2) complexity of gradient descent and provides the additional second-order guarantee that λmin(∇2f(x)) −1/2 for the computed x. Furthermore, our method is Hessian free, i.e., it only requires gradient computations, and is therefore suitable for large-scale applications.

Original languageEnglish
Pages (from-to)1751-1772
Number of pages22
JournalSIAM Journal on Optimization
Volume28
Issue number2
DOIs
StatePublished - 2018
Externally publishedYes

Funding

FundersFunder number
SAIL-Toyota
Stanford Graduate Fellowship
National Science FoundationNSF-CAREER-1553086
National Science Foundation1553086

    Keywords

    • Accelerated gradient descent
    • Convergence rate
    • Lanczos method
    • Negative curvature
    • Nonlinear optimization
    • Second-order stationarity
    • Semiconvexity

    Fingerprint

    Dive into the research topics of 'Accelerated methods for nonconvex optimization'. Together they form a unique fingerprint.

    Cite this