Coordinate methods for matrix games

Yair Carmon, Yujia Jin, Aaron Sidford, Kevin Tian

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

14 Scopus citations

Abstract

We develop primal-dual coordinate methods for solving bilinear saddle-point problems of the form min nolimits {x in mathcal{X}} max nolimits {y in mathcal{Y}}y{top}Ax which contain linear programming, classification, and regression as special cases. Our methods push existing fully stochastic sublinear methods and variance-reduced methods towards their limits in terms of per-iteration complexity and sample complexity. We obtain nearly-constant per-iteration complexity by designing efficient data structures leveraging Taylor approximations to the exponential and a binomial heap. We improve sample complexity via low-variance gradient estimators using dynamic sampling distributions that depend on both the iterates and the magnitude of the matrix entries. Our runtime bounds improve upon those of existing primal-dual methods by a factor depending on sparsity measures of the m by n matrix A. For example, when rows and columns have constant ell {1} ell {2} norm ratios, we offer improvements by a factor of m+n in the fully stochastic setting and sqrt{m+n} in the variance-reduced setting. We apply our methods to computational geometry problems, i.e. minimum enclosing ball, maximum inscribed ball, and linear regression, and obtain improved complexity bounds. For linear regression with an elementwise nonnegative matrix, our guarantees improve on exact gradient methods by a factor of sqrt{text{nnz}(A)(m+n)}.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE 61st Annual Symposium on Foundations of Computer Science, FOCS 2020
PublisherIEEE Computer Society
Pages283-293
Number of pages11
ISBN (Electronic)9781728196213
DOIs
StatePublished - Nov 2020
Event61st IEEE Annual Symposium on Foundations of Computer Science, FOCS 2020 - Virtual, Durham, United States
Duration: 16 Nov 202019 Nov 2020

Publication series

NameProceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS
Volume2020-November
ISSN (Print)0272-5428

Conference

Conference61st IEEE Annual Symposium on Foundations of Computer Science, FOCS 2020
Country/TerritoryUnited States
CityVirtual, Durham
Period16/11/2019/11/20

Funding

FundersFunder number
National Science FoundationDGE-1656518, CCF-1844855

    Keywords

    • linear regression
    • matrix games
    • minimax optimization
    • stochastic gradient methods

    Fingerprint

    Dive into the research topics of 'Coordinate methods for matrix games'. Together they form a unique fingerprint.

    Cite this