Improving training in the vicinity of temporary minima

Ido Roth, Michael Margaliot*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

An important problem in learning using gradient descent algorithms (such as backprop) is the slowdown incurred by temporary minima (TM). We consider this problem for an artificial neural network trained to solve the XOR problem. The network is transformed into the equivalent all permutations fuzzy rule-base which provides a symbolic representation of the knowledge embedded in the network. We develop a mathematical model for the evolution of the fuzzy rule-base parameters during learning in the vicinity of TM. We show that the rule-base becomes singular and tends to remain singular in the vicinity of TM. Our analysis suggests a simple remedy for overcoming the slowdown in the learning process incurred by TM. This is based on slightly perturbing the values of the training examples, so that they are no longer symmetric. Simulations demonstrate the usefulness of this approach.

Original languageEnglish
Title of host publicationBio-Inspired Systems
Subtitle of host publicationComputational and Ambient Intelligence - 10th International Work-Conference on Artificial Neural Networks, IWANN 2009, Proceedings
Pages131-139
Number of pages9
EditionPART 1
DOIs
StatePublished - 2009
Event10th International Work-Conference on Artificial Neural Networks, IWANN 2009 - Salamanca, Spain
Duration: 10 Jun 200912 Jun 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume5517 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference10th International Work-Conference on Artificial Neural Networks, IWANN 2009
Country/TerritorySpain
CitySalamanca
Period10/06/0912/06/09

Fingerprint

Dive into the research topics of 'Improving training in the vicinity of temporary minima'. Together they form a unique fingerprint.

Cite this