Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees

Haim Avron; Michael Kapralov; Cameron Musco; Christopher Musco; Ameya Velingker; Amir Zandieh

Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees

Haim Avron^*, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, Amir Zandieh

^*Corresponding author for this work

School of Mathematical Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

67 Scopus citations

Abstract

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectra) matrix approximation bounds imply statistical guarantees for kernel ridge regression.

Original language	English
Title of host publication	34th International Conference on Machine Learning, ICML 2017
Publisher	International Machine Learning Society (IMLS)
Pages	370-404
Number of pages	35
ISBN (Electronic)	9781510855144
State	Published - 2017
Event	34th International Conference on Machine Learning, ICML 2017 - Sydney, Australia Duration: 6 Aug 2017 → 11 Aug 2017

Publication series

Name	34th International Conference on Machine Learning, ICML 2017
Volume	1

Conference

Conference	34th International Conference on Machine Learning, ICML 2017
Country/Territory	Australia
City	Sydney
Period	6/08/17 → 11/08/17

Cite this

Avron, H., Kapralov, M., Musco, C., Musco, C., Velingker, A., & Zandieh, A. (2017). Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees. In 34th International Conference on Machine Learning, ICML 2017 (pp. 370-404). (34th International Conference on Machine Learning, ICML 2017; Vol. 1). International Machine Learning Society (IMLS).

@inproceedings{58dd45306a904b5b98918de8e3ebf312,

title = "Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees",

abstract = "Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectra) matrix approximation bounds imply statistical guarantees for kernel ridge regression.",

author = "Haim Avron and Michael Kapralov and Cameron Musco and Christopher Musco and Ameya Velingker and Amir Zandieh",

note = "Publisher Copyright: {\textcopyright} 2017 International Machine Learning Society (IMLS). All rights reserved.; 34th International Conference on Machine Learning, ICML 2017 ; Conference date: 06-08-2017 Through 11-08-2017",

year = "2017",

language = "אנגלית",

series = "34th International Conference on Machine Learning, ICML 2017",

publisher = "International Machine Learning Society (IMLS)",

pages = "370--404",

booktitle = "34th International Conference on Machine Learning, ICML 2017",

}

Avron, H, Kapralov, M, Musco, C, Musco, C, Velingker, A & Zandieh, A 2017, Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees. in 34th International Conference on Machine Learning, ICML 2017. 34th International Conference on Machine Learning, ICML 2017, vol. 1, International Machine Learning Society (IMLS), pp. 370-404, 34th International Conference on Machine Learning, ICML 2017, Sydney, Australia, 6/08/17.

Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees. / Avron, Haim; Kapralov, Michael; Musco, Cameron et al.
34th International Conference on Machine Learning, ICML 2017. International Machine Learning Society (IMLS), 2017. p. 370-404 (34th International Conference on Machine Learning, ICML 2017; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Random Fourier features for kernel ridge regression

T2 - 34th International Conference on Machine Learning, ICML 2017

AU - Avron, Haim

AU - Kapralov, Michael

AU - Musco, Cameron

AU - Musco, Christopher

AU - Velingker, Ameya

AU - Zandieh, Amir

PY - 2017

Y1 - 2017

N2 - Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectra) matrix approximation bounds imply statistical guarantees for kernel ridge regression.

AB - Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectra) matrix approximation bounds imply statistical guarantees for kernel ridge regression.

UR - http://www.scopus.com/inward/record.url?scp=85046992337&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85046992337

T3 - 34th International Conference on Machine Learning, ICML 2017

SP - 370

EP - 404

BT - 34th International Conference on Machine Learning, ICML 2017

PB - International Machine Learning Society (IMLS)

Y2 - 6 August 2017 through 11 August 2017

ER -

Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this