A simplified view of first order methods for optimization

Marc Teboulle

doi:10.1007/s10107-018-1284-2

A simplified view of first order methods for optimization

Marc Teboulle^*

^*Corresponding author for this work

School of Mathematical Sciences

Research output: Contribution to journal › Article › peer-review

Abstract

We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.

Original language	English
Pages (from-to)	67-96
Number of pages	30
Journal	Mathematical Programming
Volume	170
Issue number	1
DOIs	https://doi.org/10.1007/s10107-018-1284-2
State	Published - 1 Jul 2018

Keywords

Convex and nonconvex minimization
Descent Lemma
First order algorithms
Kurdyka–Łosiajewicz property
Non-Euclidean Bregman distance
Proximal framework

Access to Document

10.1007/s10107-018-1284-2

Cite this

@article{9e1da8a453e4435ea91665cb336ec247,

title = "A simplified view of first order methods for optimization",

abstract = "We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.",

keywords = "Convex and nonconvex minimization, Descent Lemma, First order algorithms, Kurdyka–{\L}osiajewicz property, Non-Euclidean Bregman distance, Proximal framework",

author = "Marc Teboulle",

note = "Publisher Copyright: {\textcopyright} 2018, Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society.",

year = "2018",

month = jul,

day = "1",

doi = "10.1007/s10107-018-1284-2",

language = "אנגלית",

volume = "170",

pages = "67--96",

journal = "Mathematical Programming",

issn = "0025-5610",

publisher = "Springer-Verlag GmbH and Co. KG",

number = "1",

}

TY - JOUR

T1 - A simplified view of first order methods for optimization

AU - Teboulle, Marc

PY - 2018/7/1

Y1 - 2018/7/1

N2 - We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.

AB - We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.

KW - Convex and nonconvex minimization

KW - Descent Lemma

KW - First order algorithms

KW - Kurdyka–Łosiajewicz property

KW - Non-Euclidean Bregman distance

KW - Proximal framework

UR - http://www.scopus.com/inward/record.url?scp=85046761883&partnerID=8YFLogxK

U2 - 10.1007/s10107-018-1284-2

DO - 10.1007/s10107-018-1284-2

M3 - מאמר

AN - SCOPUS:85046761883

VL - 170

SP - 67

EP - 96

JO - Mathematical Programming

JF - Mathematical Programming

SN - 0025-5610

IS - 1

ER -

A simplified view of first order methods for optimization

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this