On the complexity of learning with kernels

Nicolò Cesa-Bianchi, Yishay Mansour, Ohad Shamir

Research output: Contribution to journalConference articlepeer-review


A well-recognized limitation of kernel learning is the requirement to handle a kernel matrix, whose size is quadratic in the number of training examples. Many methods have been proposed to reduce this computational cost, mostly by using a subset of the kernel matrix entries, or some form of lowrank matrix approximation, or a random projection method. In this paper, we study lower bounds on the error attainable by such methods as a function of the number of entries observed in the kernel matrix or the rank of an approximate kernel matrix. We show that there are kernel learning problems where no such method will lead to non-trivial computational savings. Our results also quantify how the problem difficulty depends on parameters such as the nature of the loss function, the regularization parameter, the norm of the desired predictor, and the kernel matrix rank. Our results also suggest cases where more efficient kernel learning might be possible.

Original languageEnglish
JournalJournal of Machine Learning Research
Issue number2015
StatePublished - 2015
Event28th Conference on Learning Theory, COLT 2015 - Paris, France
Duration: 2 Jul 20156 Jul 2015


Dive into the research topics of 'On the complexity of learning with kernels'. Together they form a unique fingerprint.

Cite this