TY - GEN
T1 - Coordinate methods for matrix games
AU - Carmon, Yair
AU - Jin, Yujia
AU - Sidford, Aaron
AU - Tian, Kevin
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/11
Y1 - 2020/11
N2 - We develop primal-dual coordinate methods for solving bilinear saddle-point problems of the form min nolimits {x in mathcal{X}} max nolimits {y in mathcal{Y}}y{top}Ax which contain linear programming, classification, and regression as special cases. Our methods push existing fully stochastic sublinear methods and variance-reduced methods towards their limits in terms of per-iteration complexity and sample complexity. We obtain nearly-constant per-iteration complexity by designing efficient data structures leveraging Taylor approximations to the exponential and a binomial heap. We improve sample complexity via low-variance gradient estimators using dynamic sampling distributions that depend on both the iterates and the magnitude of the matrix entries. Our runtime bounds improve upon those of existing primal-dual methods by a factor depending on sparsity measures of the m by n matrix A. For example, when rows and columns have constant ell {1} ell {2} norm ratios, we offer improvements by a factor of m+n in the fully stochastic setting and sqrt{m+n} in the variance-reduced setting. We apply our methods to computational geometry problems, i.e. minimum enclosing ball, maximum inscribed ball, and linear regression, and obtain improved complexity bounds. For linear regression with an elementwise nonnegative matrix, our guarantees improve on exact gradient methods by a factor of sqrt{text{nnz}(A)(m+n)}.
AB - We develop primal-dual coordinate methods for solving bilinear saddle-point problems of the form min nolimits {x in mathcal{X}} max nolimits {y in mathcal{Y}}y{top}Ax which contain linear programming, classification, and regression as special cases. Our methods push existing fully stochastic sublinear methods and variance-reduced methods towards their limits in terms of per-iteration complexity and sample complexity. We obtain nearly-constant per-iteration complexity by designing efficient data structures leveraging Taylor approximations to the exponential and a binomial heap. We improve sample complexity via low-variance gradient estimators using dynamic sampling distributions that depend on both the iterates and the magnitude of the matrix entries. Our runtime bounds improve upon those of existing primal-dual methods by a factor depending on sparsity measures of the m by n matrix A. For example, when rows and columns have constant ell {1} ell {2} norm ratios, we offer improvements by a factor of m+n in the fully stochastic setting and sqrt{m+n} in the variance-reduced setting. We apply our methods to computational geometry problems, i.e. minimum enclosing ball, maximum inscribed ball, and linear regression, and obtain improved complexity bounds. For linear regression with an elementwise nonnegative matrix, our guarantees improve on exact gradient methods by a factor of sqrt{text{nnz}(A)(m+n)}.
KW - linear regression
KW - matrix games
KW - minimax optimization
KW - stochastic gradient methods
UR - http://www.scopus.com/inward/record.url?scp=85097918080&partnerID=8YFLogxK
U2 - 10.1109/FOCS46700.2020.00035
DO - 10.1109/FOCS46700.2020.00035
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85097918080
T3 - Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS
SP - 283
EP - 293
BT - Proceedings - 2020 IEEE 61st Annual Symposium on Foundations of Computer Science, FOCS 2020
PB - IEEE Computer Society
T2 - 61st IEEE Annual Symposium on Foundations of Computer Science, FOCS 2020
Y2 - 16 November 2020 through 19 November 2020
ER -