Linear Independent Component Analysis over Finite Fields: Algorithms and Bounds

Amichai Painsky*, Saharon Rosset, Meir Feder

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Independent component analysis (ICA) is a statistical tool that decomposes an observed random vector into components that are as statistically independent as possible. ICA over finite fields is a special case of ICA, in which both the observations and the decomposed components take values over a finite alphabet. This problem is also known as minimal redundancy representation or factorial coding. In this paper, we focus on linear methods for ICA over finite fields. We introduce a basic lower bound that provides a fundamental limit to the ability of any linear solution to solve this problem. Based on this bound, we present a greedy algorithm that outperforms all currently known methods. Importantly, we show that the overhead of our suggested algorithm (compared with the lower bound) typically decreases as the scale of the problem grows. In addition, we provide a sub-optimal variant of our suggested method that significantly reduces the computational complexity at a relatively small cost in performance. Finally, we discuss the universal abilities of linear transformations in decomposing random vectors, compared with existing non-linear solutions.

Original languageEnglish
Article number8471206
Pages (from-to)5875-5886
Number of pages12
JournalIEEE Transactions on Signal Processing
Volume66
Issue number22
DOIs
StatePublished - 15 Nov 2018

Keywords

  • Independent component analysis
  • binary ICA
  • blind source separation
  • factorial codes
  • minimal redundancy representation
  • minimum entropy codes

Fingerprint

Dive into the research topics of 'Linear Independent Component Analysis over Finite Fields: Algorithms and Bounds'. Together they form a unique fingerprint.

Cite this