Most informative dimension reduction

Amir Globerson, Naftali Tishby

Research output: Contribution to conferencePaperpeer-review

Abstract

Finding effective low dimensional features from empirical co-occurrence data is one of the most fundamental problems in machine learning and complex data analysis. One principled approach to this problem is to represent the data in low dimension with minimal loss of the information contained in the original data. In this paper we present a novel information theoretic principle and algorithm for extracting low dimensional representations, or feature-vectors, that capture as much as possible of the mutual information between the variables. Unlike previous work in this direction, here we do not cluster or quantize the variables, but rather extract continuous feature functions directly from the co-occurrence matrix, using a converging iterative projection algorithm. The obtained features serve, in a well defined way, as approximate sufficient statistics that capture the information in a joint sample of the variables. Our approach is both simpler and more general than clustering or mixture models and is applicable to a wide range of problems, from document categorization to bioinformatics and analysis of neural codes.

Original languageEnglish
Pages1024-1029
Number of pages6
StatePublished - 2002
Externally publishedYes
Event18th National Conference on Artificial Intelligence (AAAI-02), 14th Innovative Applications of Artificial Intelligence Conference (IAAI-02) - Edmonton, Alta., Canada
Duration: 28 Jul 20021 Aug 2002

Conference

Conference18th National Conference on Artificial Intelligence (AAAI-02), 14th Innovative Applications of Artificial Intelligence Conference (IAAI-02)
Country/TerritoryCanada
CityEdmonton, Alta.
Period28/07/021/08/02

Fingerprint

Dive into the research topics of 'Most informative dimension reduction'. Together they form a unique fingerprint.

Cite this