Lower bounds for finding stationary points II: first-order methods

Yair Carmon*, John C. Duchi, Oliver Hinder, Aaron Sidford

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

26 Scopus citations

Abstract

We establish lower bounds on the complexity of finding ϵ-stationary points of smooth, non-convex high-dimensional functions using first-order methods. We prove that deterministic first-order methods, even applied to arbitrarily smooth functions, cannot achieve convergence rates in ϵ better than ϵ- 8 / 5, which is within ϵ-1/15log1ϵ of the best known rate for such methods. Moreover, for functions with Lipschitz first and second derivatives, we prove that no deterministic first-order method can achieve convergence rates better than ϵ- 12 / 7, while ϵ- 2 is a lower bound for functions with only Lipschitz gradient. For convex functions with Lipschitz gradient, accelerated gradient descent achieves a better rate, showing that finding stationary points is easier given convexity.

Original languageEnglish
Pages (from-to)315-355
Number of pages41
JournalMathematical Programming
Volume185
Issue number1-2
DOIs
StatePublished - Jan 2021
Externally publishedYes

Funding

FundersFunder number
National Science Foundation1844855, CCF-1844855
Division of Computing and Communication Foundations1553086
Center for Selective C-H Functionalization, National Science Foundation
Center for Hierarchical Manufacturing, National Science Foundation

    Keywords

    • Accelerated gradient descent
    • Dimension-free rates
    • Gradient methods
    • Information-based complexity
    • Non-convex optimization

    Fingerprint

    Dive into the research topics of 'Lower bounds for finding stationary points II: first-order methods'. Together they form a unique fingerprint.

    Cite this