We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.
- Convex and nonconvex minimization
- Descent Lemma
- First order algorithms
- Kurdyka–Łosiajewicz property
- Non-Euclidean Bregman distance
- Proximal framework