Wavelet decompositions of Random Forests-smoothness analysis, sparse approximation and applications

Oren Elisha, Shai Dekel

Research output: Contribution to journalArticlepeer-review


In this paper we introduce, in the setting of machine learning, a generalization of wavelet analysis which is a popular approach to low dimensional structured signal analysis. The wavelet decomposition of a Random Forest provides a sparse approximation of any regression or classification high dimensional function at various levels of detail, with a concrete ordering of the Random Forest nodes: from 'significant' elements to nodes capturing only 'insignificant' noise. Motivated by function space theory, we use the wavelet decomposition to compute numerically a 'weak-Type' smoothness index that captures the complexity of the underlying function. As we show through extensive experimentation, this sparse representation facilitates a variety of applications such as improved regression for difficult datasets, a novel approach to feature importance, resilience to noisy or irrelevant features, compression of ensembles, etc.

Original languageEnglish
Pages (from-to)1-38
Number of pages38
JournalJournal of Machine Learning Research
StatePublished - 1 Nov 2016


  • Adaptive approximation
  • Besov spaces
  • Feature importance.
  • Random Forest
  • Wavelets


Dive into the research topics of 'Wavelet decompositions of Random Forests-smoothness analysis, sparse approximation and applications'. Together they form a unique fingerprint.

Cite this