High-Performance Kernel Machines With Implicit Distributed Optimization and Randomization

Haim Avron*, Vikas Sindhwani

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


We propose a framework for massive-scale training of kernel-based statistical models, based on combining distributed convex optimization with randomization techniques. Our approach is based on a block-splitting variant of the alternating directions method of multipliers, carefully reconfigured to handle very large random feature matrices under memory constraints, while exploiting hybrid parallelism typically found in modern clusters of multicore machines. Our high-performance implementation supports a variety of statistical learning tasks by enabling several loss functions, regularization schemes, kernels, and layers of randomized approximations for both dense and sparse datasets, in an extensible framework. We evaluate our implementation on large-scale model construction tasks and provide a comparison against existing sequential and parallel libraries. Supplementary materials for this article are available online.

Original languageEnglish
Pages (from-to)341-349
Number of pages9
Issue number3
StatePublished - 2 Jul 2016


FundersFunder number
Defense Advanced Research Projects AgencyFA8750-12-C-0323
International Business Machines Corporation


    • Big-data
    • Kernel methods
    • Scalability
    • Statistical computations


    Dive into the research topics of 'High-Performance Kernel Machines With Implicit Distributed Optimization and Randomization'. Together they form a unique fingerprint.

    Cite this