Improving stability of decision trees

Mark Last*, Oded Maimon, Einat Minkov

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

46 Scopus citations

Abstract

Decision-tree algorithms are known to be unstable: small variations in the training set can result in different trees and different predictions for the same validation examples. Both accuracy and stability can be improved by learning multiple models from boot-strap samples of training data, but the "meta-learner" approach makes the extracted knowledge hardly interpretable. In the following paper, we present the Info-Fuzzy Network (IFN), a novel information-theoretic method for building stable and comprehensible decision-tree models. The stability of the IFN algorithm is ensured by restricting the tree structure to using the same feature for all nodes of the same tree level and by the built-in statistical significance tests. The IFN method is shown empirically to produce more compact and stable models than the "meta-learner" techniques, while preserving a reasonable level of predictive accuracy.

Original languageEnglish
Pages (from-to)145-159
Number of pages15
JournalInternational Journal of Pattern Recognition and Artificial Intelligence
Volume16
Issue number2
DOIs
StatePublished - Mar 2002

Keywords

  • Classification accuracy
  • Decision trees
  • Info-fuzzy network
  • Multiple models
  • Output complexity
  • Similarity
  • Stability

Fingerprint

Dive into the research topics of 'Improving stability of decision trees'. Together they form a unique fingerprint.

Cite this