Generalized independent component analysis over finite alphabets

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Independent component analysis (ICA) is a statistical method for transforming an observable multi-dimensional random vector into components that are as statistically independent as possible from each other. Usually, the ICA framework assumes a model according to which the observations are generated (such as a linear transformation with additive noise). ICA over finite fields is a special case of ICA in which both the observations and the independent components are over a finite alphabet. In this paper, we consider a generalization of this framework in which an observation vector is decomposed to its independent components (as much as possible) with no prior assumption on the way it was generated. This generalization is also known as Barlow's minimal redundancy representation problem and is considered an open problem. We propose several theorems and show that this hard problem can be accurately solved with a branch and bound search tree algorithm, or tightly approximated with a series of linear problems. Our contribution provides the first efficient set of solutions to Barlow's problem. The minimal redundancy representation (also known as factorial code) has many applications, mainly in the fields of neural networks and deep learning. The binary ICA is also shown to have applications in several domains, including medical diagnosis, multi-cluster assignment, network tomography, and internet resource management. In this paper, we show that this formulation further applies to multiple disciplines in source coding, such as predictive coding, distributed source coding, and coding of large alphabet sources.

Original languageEnglish
Article number2510657
Pages (from-to)1038-1053
Number of pages16
JournalIEEE Transactions on Information Theory
Issue number2
StatePublished - 1 Feb 2016


  • BICA
  • Blind source separation
  • Distributed source coding
  • Factorial codes
  • ICA over galois field
  • Independent component analysis
  • Minimal redundancy representation
  • Minimum entropy codes
  • Neural networks
  • Predictive coding


Dive into the research topics of 'Generalized independent component analysis over finite alphabets'. Together they form a unique fingerprint.

Cite this