A new approach to knowledge-based design of recurrent neural networks

Eyal Kolman, Michael Margaliot

Research output: Contribution to journalArticlepeer-review

Abstract

A major drawback of artificial neural networks (ANNs) is their black-box character. This is especially true for recurrent neural networks (RNNs) because of their intricate feedback connections. In particular, given a problem and some initial information concerning its solution, it is not at all obvious how to design an RNN that is suitable for solving this problem. In this paper, we consider a fuzzy rule base with a special structure, referred to as the fuzzy all-permutations rule base (FARB). Inferring the FARB yields an input-output (IO) mapping that is mathematically equivalent to that of an RNN. We use this equivalence to develop two new knowledge-based design methods for RNNs. The first method, referred to as the direct approach, is based on stating the desired functioning of the RNN in terms of several sets of symbolic rules, each one corresponding to a subnetwork. Each set is then transformed into a suitable FARB. The second method is based on first using the direct approach to design a library of simple modules, such as counters or comparators, and realize them using RNNs. Once designed, the correctness of each RNN can be verified. Then, the initial design problem is solved by using these basic modules as building blocks. This yields a modular and systematic approach for knowledge-based design of RNNs. We demonstrate the efficiency of these approaches by designing RNNs that recognize both regular and nonregular formal languages.

Original languageEnglish
Pages (from-to)1389-1401
Number of pages13
JournalIEEE Transactions on Neural Networks
Volume19
Issue number8
DOIs
StatePublished - 2008

Keywords

  • Context-free grammar
  • Formal languages
  • Fuzzy all-permutations rule base (FARB)
  • Knowledge insertion
  • Knowledge-based neurocomputing
  • Neurofuzzy systems
  • Recurrent neural networks (RNNs)
  • Regular grammar

Fingerprint

Dive into the research topics of 'A new approach to knowledge-based design of recurrent neural networks'. Together they form a unique fingerprint.

Cite this