Past-future information bottleneck in dynamical systems

Felix Creutzig, Amir Globerson, Naftali Tishby

Research output: Contribution to journalArticlepeer-review

Abstract

Biological systems need to process information in real time and must trade off accuracy of presentation and coding costs. Here we operationalize this trade-off and develop an information-theoretic framework that selectively extracts information of the input past that is predictive about the output future, obtaining a generalized eigenvalue problem. Thereby, we unravel the input history in terms of structural phase transitions corresponding to additional dimensions of a state space. We elucidate the relation to canonical correlation analysis and give a numerical example. Altogether, this work relates information-theoretic optimization to the joint problem of system identification and model reduction.

Original languageEnglish
Article number041925
JournalPhysical Review E - Statistical, Nonlinear, and Soft Matter Physics
Volume79
Issue number4
DOIs
StatePublished - 27 Apr 2009
Externally publishedYes

Fingerprint

Dive into the research topics of 'Past-future information bottleneck in dynamical systems'. Together they form a unique fingerprint.

Cite this