Hierarchically compositional kernels for scalable nonparametric learning

Jie Chen, Haim Avron, Vikas Sindhwani

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods. The proposed kernel is defined based on a hierarchical partitioning of the underlying data domain, where the Nyström method (a globally low-rank approximation) is married with a locally lossless approximation in a hierarchical fashion. The kernel maintains (strict) positive-definiteness. The corresponding kernel matrix admits a recursively off-diagonal low-rank structure, which allows for fast linear algebra computations. Suppressing the factor of data dimension, the memory and arithmetic complexities for training a regression or a classifier are reduced from O(n2) and O(n3) to O(nr) and O(nr2), respectively, where n is the number of training examples and r is the rank on each level of the hierarchy. Although other randomized approximate kernels entail a similar complexity, empirical results show that the proposed kernel achieves a matching performance with a smaller r. We demonstrate comprehensive experiments to show the effective use of the proposed kernel on data sizes up to the order of millions.

Original languageEnglish
Pages (from-to)1-42
Number of pages42
JournalJournal of Machine Learning Research
Volume18
StatePublished - 1 Aug 2017

Funding

FundersFunder number
Defense Advanced Research Projects Agency
Air Force Research LaboratoryFA8750-12-C-0323

    Keywords

    • Hierarchical kernels
    • Nonparametric learning

    Fingerprint

    Dive into the research topics of 'Hierarchically compositional kernels for scalable nonparametric learning'. Together they form a unique fingerprint.

    Cite this