## Abstract

We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form e^{A}b, which the Lanczos method approximates efficiently. Our key technique is to view the sketch as a randomized mirror projection, and perform mirror descent analysis on the expected projection. Our sketch solves the online eigenvector problem, improving the best known complexity bounds by Ω(log^{5}n). We also apply this sketch to semidefinite programming in saddle-point form, yielding a simple primal-dual scheme with guarantees matching the best in the literature.

Original language | English |
---|---|

Pages (from-to) | 589-623 |

Number of pages | 35 |

Journal | Proceedings of Machine Learning Research |

Volume | 99 |

State | Published - 2019 |

Externally published | Yes |

Event | 32nd Conference on Learning Theory, COLT 2019 - Phoenix, United States Duration: 25 Jun 2019 → 28 Jun 2019 |

### Funding

Funders | Funder number |
---|---|

National Science Foundation | 1553086 |

Alfred P. Sloan Foundation | DGE-1656518, ONR-YIP N00014-19-1-2288, CCF-1844855 |

## Keywords

- Lanczos method
- Online learning
- matrix exponential
- mirror descent
- spectrahedron