Optimal ensemble averaging of neural networks

Research output: Contribution to journalArticlepeer-review

Abstract

Based on an observation about the different effect of ensemble averaging on the bias and variance portions of the prediction error, we discuss training methodologies for ensembles of networks. We demonstrate the effect of variance reduction and present a method of extrapolation to the limit of an infinite ensemble. A significant reduction of variance is obtained by averaging just over initial conditions of the neural networks, without varying architectures or training sets. The minimum of the ensemble prediction error is reached later than that of a single network. In the vicinity of the minimum, the ensemble prediction error appears to be flatter than that of the single network, thus simplifying optimal stopping decision. The results are demonstrated on sunspots data, where the predictions are among the best obtained, and on the 1993 energy prediction competition data set B.

Original languageEnglish
Pages (from-to)283-296
Number of pages14
JournalNetwork: Computation in Neural Systems
Volume8
Issue number3
DOIs
StatePublished - Aug 1997

Fingerprint

Dive into the research topics of 'Optimal ensemble averaging of neural networks'. Together they form a unique fingerprint.

Cite this