An Accelerated Coordinate Gradient Descent Algorithm for Non-separable Composite Optimization

Aviad Aberdam, Amir Beck

Research output: Contribution to journalArticlepeer-review

Abstract

Coordinate descent algorithms are popular in machine learning and large-scale data analysis problems due to their low computational cost iterative schemes and their improved performances. In this work, we define a monotone accelerated coordinate gradient descent-type method for problems consisting of minimizing f+ g, where f is quadratic and g is nonsmooth and non-separable and has a low-complexity proximal mapping. The algorithm is enabled by employing the forward–backward envelope, a composite envelope that possess an exact smooth reformulation of f+ g. We prove the algorithm achieves a convergence rate of O(1 / k1.5) in terms of the original objective function, improving current coordinate descent-type algorithms. In addition, we describe an adaptive variant of the algorithm that backtracks the spectral information and coordinate Lipschitz constants of the problem. We numerically examine our algorithms on various settings, including two-dimensional total-variation-based image inpainting problems, showing a clear advantage in performance over current coordinate descent-type methods.

Original languageEnglish
JournalJournal of Optimization Theory and Applications
DOIs
StateAccepted/In press - 2021

Keywords

  • Composite functions
  • Convex optimization
  • Coordinate gradient descent
  • Forward–backward envelope

Fingerprint

Dive into the research topics of 'An Accelerated Coordinate Gradient Descent Algorithm for Non-separable Composite Optimization'. Together they form a unique fingerprint.

Cite this