Kernel Feature Selection with Side Data Using a Spectral Approach

Amnon Shashua, Lior Wolf

Research output: Contribution to journalArticlepeer-review

Abstract

We address the problem of selecting a subset of the most relevant features from a set of sample data in cases where there are multiple (equally reasonable) solutions. In particular, this topic includes on one hand the introduction of hand-crafted kernels which emphasize certain desirable aspects of the data and, on the other hand, the suppression of one of the solutions given "side" data, i.e., when one is given information about undesired aspects of the data. Such situations often arise when there are several, even conflicting, dimensions to the data. For example, documents can be clustered based on topic, authorship or writing style; images of human faces can be clustered based on illumination conditions, facial expressions or by person identity, and so forth. Starting from a spectral method for feature selection, known as Q - α, we introduce first a kernel version of the approach thereby adding the power of non-linearity to the underlying representations and the choice to emphasize certain kernel-dependent aspects of the data. As an alternative to the use of a kernel we introduce a principled manner for making use of auxiliary data within a spectral approach for handling situations where multiple subsets of relevant features exist in the data. The algorithm we will introduce allows for inhibition of relevant features of the auxiliary dataset and allows for creating a topological model of all relevant feature subsets in the dataset. To evaluate the effectiveness of our approach we have conducted experiments both on real-images of human faces under varying illumination, facial expressions and person identity and on general machine learning tasks taken from the UC Irvine repository. The performance of our algorithm for selecting features with side information is generally superior to current methods we tested (PCA.OPCA.CPCA and SDR-SI).

Original languageEnglish
Pages (from-to)39-53
Number of pages15
JournalLecture Notes in Computer Science
Volume3023
DOIs
StatePublished - 2004
Externally publishedYes

Fingerprint

Dive into the research topics of 'Kernel Feature Selection with Side Data Using a Spectral Approach'. Together they form a unique fingerprint.

Cite this