Extracting symbolic knowledge from recurrent neural networks-A fuzzy logic approach

Eyal Kolman, Michael Margaliot

Research output: Contribution to journalArticlepeer-review


Considerable research has been devoted to the integration of fuzzy logic (FL) tools with classic artificial intelligence (AI) paradigms. One reason for this is that FL provides powerful mechanisms for handling and processing symbolic information stated using natural language. In this respect, fuzzy rule-based systems are white-boxes, as they process information in a form that is easy to understand, verify and, if necessary, refine. The synergy between artificial neural networks (ANNs), which are notorious for their black-box character, and FL proved to be particularly successful. Such a synergy allows combining the powerful learning-from-examples capability of ANNs with the high-level symbolic information processing of FL systems. In this paper, we present a new approach for extracting symbolic information from recurrent neural networks (RNNs). The approach is based on the mathematical equivalence between a specific fuzzy rule-base and functions composed of sums of sigmoids. We show that this equivalence can be used to provide a comprehensible explanation of the RNN functioning. We demonstrate the applicability of our approach by using it to extract the knowledge embedded within an RNN trained to recognize a formal language.

Original languageEnglish
Pages (from-to)145-161
Number of pages17
JournalFuzzy Sets and Systems
Issue number2
StatePublished - 16 Jan 2009


  • All permutations fuzzy rule-base
  • Formal language
  • Hybrid intelligent systems
  • Knowledge extraction
  • Knowledge-based neurocomputing
  • Neuro-fuzzy systems
  • Recurrent neural networks
  • Regular grammar
  • Rule extraction
  • Rule generation


Dive into the research topics of 'Extracting symbolic knowledge from recurrent neural networks-A fuzzy logic approach'. Together they form a unique fingerprint.

Cite this