First-order methods for nonconvex quadratic minimization

Yair Carmon, John C. Duchi

Research output: Contribution to journalArticlepeer-review

Abstract

We consider minimization of indefinite quadratics with either trust-region (norm) constraints or cubic regularization. Despite the nonconvexity of these problems we prove that, under mild assumptions, gradient descent converges to their global solutions and give a nonasymptotic rate of convergence for the cubic variant. We also consider Krylov subspace solutions and establish sharp convergence guarantees to the solutions of both trust-region and cubic-regularized problems. Our rates mirror the behavior of these methods on convex quadratics and eigenvector problems, highlighting their scalability. When we use Krylov subspace solutions to approximate the cubic-regularized Newton step, our results recover the strongest known convergence guarantees to approximate second-order stationary points of general smooth nonconvex functions.

Original languageEnglish
Pages (from-to)395-436
Number of pages42
JournalSIAM Review
Volume62
Issue number2
DOIs
StatePublished - 2020
Externally publishedYes

Keywords

  • Cubic regularization
  • Global optimization
  • Gradient descent
  • Krylov subspace methods
  • Newton's method
  • Nonasymptotic convergence
  • Nonconvex quadratics
  • Trust-region methods

Fingerprint

Dive into the research topics of 'First-order methods for nonconvex quadratic minimization'. Together they form a unique fingerprint.

Cite this