TY - GEN

T1 - Coordinate methods for matrix games

AU - Carmon, Yair

AU - Jin, Yujia

AU - Sidford, Aaron

AU - Tian, Kevin

N1 - Publisher Copyright:
© 2020 IEEE.

PY - 2020/11

Y1 - 2020/11

N2 - We develop primal-dual coordinate methods for solving bilinear saddle-point problems of the form min nolimits {x in mathcal{X}} max nolimits {y in mathcal{Y}}y{top}Ax which contain linear programming, classification, and regression as special cases. Our methods push existing fully stochastic sublinear methods and variance-reduced methods towards their limits in terms of per-iteration complexity and sample complexity. We obtain nearly-constant per-iteration complexity by designing efficient data structures leveraging Taylor approximations to the exponential and a binomial heap. We improve sample complexity via low-variance gradient estimators using dynamic sampling distributions that depend on both the iterates and the magnitude of the matrix entries. Our runtime bounds improve upon those of existing primal-dual methods by a factor depending on sparsity measures of the m by n matrix A. For example, when rows and columns have constant ell {1} ell {2} norm ratios, we offer improvements by a factor of m+n in the fully stochastic setting and sqrt{m+n} in the variance-reduced setting. We apply our methods to computational geometry problems, i.e. minimum enclosing ball, maximum inscribed ball, and linear regression, and obtain improved complexity bounds. For linear regression with an elementwise nonnegative matrix, our guarantees improve on exact gradient methods by a factor of sqrt{text{nnz}(A)(m+n)}.

AB - We develop primal-dual coordinate methods for solving bilinear saddle-point problems of the form min nolimits {x in mathcal{X}} max nolimits {y in mathcal{Y}}y{top}Ax which contain linear programming, classification, and regression as special cases. Our methods push existing fully stochastic sublinear methods and variance-reduced methods towards their limits in terms of per-iteration complexity and sample complexity. We obtain nearly-constant per-iteration complexity by designing efficient data structures leveraging Taylor approximations to the exponential and a binomial heap. We improve sample complexity via low-variance gradient estimators using dynamic sampling distributions that depend on both the iterates and the magnitude of the matrix entries. Our runtime bounds improve upon those of existing primal-dual methods by a factor depending on sparsity measures of the m by n matrix A. For example, when rows and columns have constant ell {1} ell {2} norm ratios, we offer improvements by a factor of m+n in the fully stochastic setting and sqrt{m+n} in the variance-reduced setting. We apply our methods to computational geometry problems, i.e. minimum enclosing ball, maximum inscribed ball, and linear regression, and obtain improved complexity bounds. For linear regression with an elementwise nonnegative matrix, our guarantees improve on exact gradient methods by a factor of sqrt{text{nnz}(A)(m+n)}.

KW - linear regression

KW - matrix games

KW - minimax optimization

KW - stochastic gradient methods

UR - http://www.scopus.com/inward/record.url?scp=85097918080&partnerID=8YFLogxK

U2 - 10.1109/FOCS46700.2020.00035

DO - 10.1109/FOCS46700.2020.00035

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85097918080

T3 - Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

SP - 283

EP - 293

BT - Proceedings - 2020 IEEE 61st Annual Symposium on Foundations of Computer Science, FOCS 2020

PB - IEEE Computer Society

T2 - 61st IEEE Annual Symposium on Foundations of Computer Science, FOCS 2020

Y2 - 16 November 2020 through 19 November 2020

ER -