Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold

Boris Shustin*, Haim Avron

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear eigenvalue problems, and in electronic structures computations, to name a few problems. In statistics and machine learning, they arise, for example, in various dimensionality reduction techniques such as canonical correlation analysis. In deep learning, regularization and improved stability can be obtained by constraining some layers to have parameter matrices that belong to the Stiefel manifold. Solving problems on the generalized Stiefel manifold can be approached via the tools of Riemannian optimization. However, using the standard geometric components for the generalized Stiefel manifold has two possible shortcomings: computing some of the geometric components can be too expensive and convergence can be rather slow in certain cases. Both shortcomings can be addressed using a technique called Riemannian preconditioning, which amounts to using geometric components derived by a preconditioner that defines a Riemannian metric on the constraint manifold. In this paper we develop the geometric components required to perform Riemannian optimization on the generalized Stiefel manifold equipped with a non-standard metric, and illustrate theoretically and numerically the use of those components and the effect of Riemannian preconditioning for solving optimization problems on the generalized Stiefel manifold.

Original languageEnglish
Article number114953
JournalJournal of Computational and Applied Mathematics
Volume423
DOIs
StatePublished - 15 May 2023

Funding

FundersFunder number
Israel Science Foundation1272/17

    Keywords

    • Generalized Stiefel manifold
    • Preconditioning
    • Riemannian optimization

    Fingerprint

    Dive into the research topics of 'Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold'. Together they form a unique fingerprint.

    Cite this