Nonlinear canonical correlation analysis: A compressed representation approach

Amichai Painsky*, Meir Feder, Naftali Tishby

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Canonical Correlation Analysis (CCA) is a linear representation learning method that seeks maximally correlated variables in multi-view data. Nonlinear CCA extends this notion to a broader family of transformations, which are more powerful in many real-world applications. Given the joint probability, the Alternating Conditional Expectation (ACE) algorithm provides an optimal solution to the nonlinear CCA problem. However, it suffers from limited performance and an increasing computational burden when only a finite number of samples is available. In this work, we introduce an information-theoretic compressed representation framework for the nonlinear CCA problem (CRCCA), which extends the classical ACE approach. Our suggested framework seeks compact representations of the data that allow a maximal level of correlation. This way, we control the trade-off between the flexibility and the complexity of the model. CRCCA provides theoretical bounds and optimality conditions, as we establish fundamental connections to rate-distortion theory, the information bottleneck and remote source coding. In addition, it allows a soft dimensionality reduction, as the compression level is determined by the mutual information between the original noisy data and the extracted signals. Finally, we introduce a simple implementation of the CRCCA framework, based on lattice quantization.

Original languageEnglish
Article number208
Issue number2
StatePublished - 1 Feb 2020


FundersFunder number
Gatsby Charitable Foundation
Intel Collaboration Research Institute for Computational Intelligence


    • Alternating conditional expectation
    • Canonical correlation analysis
    • Dimensionality reduction
    • Information bottleneck
    • Remote source coding


    Dive into the research topics of 'Nonlinear canonical correlation analysis: A compressed representation approach'. Together they form a unique fingerprint.

    Cite this