A Rank-1 Sketch for Matrix Multiplicative Weights

Yair Carmon; John C. Duchi; Sidford Aaron; Tian Kevin

A Rank-1 Sketch for Matrix Multiplicative Weights

Yair Carmon, John C. Duchi, Sidford Aaron, Tian Kevin

Stanford University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a randomized mirror projection, and perform mirror descent analysis on the expected projection. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by $ n)$. We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.

Original language	English
Title of host publication	Proceedings of the Thirty-Second Conference on Learning Theory (COLT)
Editors	Alina Beygelzimer, Daniel Hsu
Place of Publication	Phoenix, USA
Publisher	PMLR
Pages	589-623
Number of pages	35
Volume	99
State	Published - 1 Jan 2019
Externally published	Yes
Event	32nd Annual Conference on Learning Theory, COLT 2019 - Phoenix, United States Duration: 25 Jun 2019 → 28 Jun 2019 Conference number: 32

Publication series

Name	Proceedings of Machine Learning Research
Publisher	PMLR
ISSN (Electronic)	2640-3498

Conference

Conference	32nd Annual Conference on Learning Theory, COLT 2019
Abbreviated title	COLT 2019
Country/Territory	United States
City	Phoenix
Period	25/06/19 → 28/06/19

Access to Document

http://proceedings.mlr.press/v99/carmon19a.html

Cite this

@inproceedings{aa4e69beb1214bcab04ff74f52a4b914,

title = "A Rank-1 Sketch for Matrix Multiplicative Weights",

abstract = "We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a randomized mirror projection, and perform mirror descent analysis on the expected projection. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by $ n)$. We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.",

author = "Yair Carmon and Duchi, {John C.} and Sidford Aaron and Tian Kevin",

year = "2019",

month = jan,

day = "1",

language = "אנגלית",

volume = "99",

series = "Proceedings of Machine Learning Research",

publisher = "PMLR",

pages = "589--623",

editor = "Alina Beygelzimer and Daniel Hsu",

booktitle = "Proceedings of the Thirty-Second Conference on Learning Theory (COLT)",

note = "32nd Annual Conference on Learning Theory, COLT 2019 ; Conference date: 25-06-2019 Through 28-06-2019",

}

Carmon, Y, Duchi, JC, Aaron, S & Kevin, T 2019, A Rank-1 Sketch for Matrix Multiplicative Weights. in A Beygelzimer & D Hsu (eds), Proceedings of the Thirty-Second Conference on Learning Theory (COLT). vol. 99, Proceedings of Machine Learning Research, PMLR, Phoenix, USA, pp. 589-623, 32nd Annual Conference on Learning Theory, COLT 2019, Phoenix, Arizona, United States, 25/06/19. <http://proceedings.mlr.press/v99/carmon19a.html>

A Rank-1 Sketch for Matrix Multiplicative Weights. / Carmon, Yair; Duchi, John C.; Aaron, Sidford et al.
Proceedings of the Thirty-Second Conference on Learning Theory (COLT). ed. / Alina Beygelzimer; Daniel Hsu. Vol. 99 Phoenix, USA: PMLR, 2019. p. 589-623 (Proceedings of Machine Learning Research).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Rank-1 Sketch for Matrix Multiplicative Weights

AU - Carmon, Yair

AU - Duchi, John C.

AU - Aaron, Sidford

AU - Kevin, Tian

N1 - Conference code: 32

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a randomized mirror projection, and perform mirror descent analysis on the expected projection. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by $ n)$. We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.

AB - We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a randomized mirror projection, and perform mirror descent analysis on the expected projection. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by $ n)$. We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

VL - 99

T3 - Proceedings of Machine Learning Research

SP - 589

EP - 623

BT - Proceedings of the Thirty-Second Conference on Learning Theory (COLT)

A2 - Beygelzimer, Alina

A2 - Hsu, Daniel

PB - PMLR

CY - Phoenix, USA

T2 - 32nd Annual Conference on Learning Theory, COLT 2019

Y2 - 25 June 2019 through 28 June 2019

ER -

A Rank-1 Sketch for Matrix Multiplicative Weights

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this