An Accelerated Coordinate Gradient Descent Algorithm for Non-separable Composite Optimization

Aviad Aberdam, Amir Beck*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Coordinate descent algorithms are popular in machine learning and large-scale data analysis problems due to their low computational cost iterative schemes and their improved performances. In this work, we define a monotone accelerated coordinate gradient descent-type method for problems consisting of minimizing f+ g, where f is quadratic and g is nonsmooth and non-separable and has a low-complexity proximal mapping. The algorithm is enabled by employing the forward–backward envelope, a composite envelope that possess an exact smooth reformulation of f+ g. We prove the algorithm achieves a convergence rate of O(1 / k1.5) in terms of the original objective function, improving current coordinate descent-type algorithms. In addition, we describe an adaptive variant of the algorithm that backtracks the spectral information and coordinate Lipschitz constants of the problem. We numerically examine our algorithms on various settings, including two-dimensional total-variation-based image inpainting problems, showing a clear advantage in performance over current coordinate descent-type methods.

Original languageEnglish
Pages (from-to)219-246
Number of pages28
JournalJournal of Optimization Theory and Applications
Issue number1-3
StatePublished - Jun 2022


FundersFunder number
Israel Science Foundation926-21
Azrieli Foundation


    • Composite functions
    • Convex optimization
    • Coordinate gradient descent
    • Forward–backward envelope


    Dive into the research topics of 'An Accelerated Coordinate Gradient Descent Algorithm for Non-separable Composite Optimization'. Together they form a unique fingerprint.

    Cite this