TY - JOUR
T1 - Linear Independent Component Analysis over Finite Fields
T2 - Algorithms and Bounds
AU - Painsky, Amichai
AU - Rosset, Saharon
AU - Feder, Meir
N1 - Publisher Copyright:
© 1991-2012 IEEE.
PY - 2018/11/15
Y1 - 2018/11/15
N2 - Independent component analysis (ICA) is a statistical tool that decomposes an observed random vector into components that are as statistically independent as possible. ICA over finite fields is a special case of ICA, in which both the observations and the decomposed components take values over a finite alphabet. This problem is also known as minimal redundancy representation or factorial coding. In this paper, we focus on linear methods for ICA over finite fields. We introduce a basic lower bound that provides a fundamental limit to the ability of any linear solution to solve this problem. Based on this bound, we present a greedy algorithm that outperforms all currently known methods. Importantly, we show that the overhead of our suggested algorithm (compared with the lower bound) typically decreases as the scale of the problem grows. In addition, we provide a sub-optimal variant of our suggested method that significantly reduces the computational complexity at a relatively small cost in performance. Finally, we discuss the universal abilities of linear transformations in decomposing random vectors, compared with existing non-linear solutions.
AB - Independent component analysis (ICA) is a statistical tool that decomposes an observed random vector into components that are as statistically independent as possible. ICA over finite fields is a special case of ICA, in which both the observations and the decomposed components take values over a finite alphabet. This problem is also known as minimal redundancy representation or factorial coding. In this paper, we focus on linear methods for ICA over finite fields. We introduce a basic lower bound that provides a fundamental limit to the ability of any linear solution to solve this problem. Based on this bound, we present a greedy algorithm that outperforms all currently known methods. Importantly, we show that the overhead of our suggested algorithm (compared with the lower bound) typically decreases as the scale of the problem grows. In addition, we provide a sub-optimal variant of our suggested method that significantly reduces the computational complexity at a relatively small cost in performance. Finally, we discuss the universal abilities of linear transformations in decomposing random vectors, compared with existing non-linear solutions.
KW - Independent component analysis
KW - binary ICA
KW - blind source separation
KW - factorial codes
KW - minimal redundancy representation
KW - minimum entropy codes
UR - http://www.scopus.com/inward/record.url?scp=85054216997&partnerID=8YFLogxK
U2 - 10.1109/TSP.2018.2872006
DO - 10.1109/TSP.2018.2872006
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:85054216997
SN - 1053-587X
VL - 66
SP - 5875
EP - 5886
JO - IEEE Transactions on Signal Processing
JF - IEEE Transactions on Signal Processing
IS - 22
M1 - 8471206
ER -