Spectral bounds for sparse PCA: Exact and greedy algorithms

Baback Moghaddam, Yair Weiss, Shai Avidan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Sparse PCA seeks approximate sparse "eigenvectors" whose projections capture the maximal variance of data. As a cardinality-constrained and non-convex optimization problem, it is NP-hard and is encountered in a wide range of applied fields, from bio-informatics to finance. Recent progress has focused mainly on continuous approximation and convex relaxation of the hard cardinality constraint. In contrast, we consider an alternative discrete spectral formulation based on variational eigenvalue bounds and provide an effective greedy strategy as well as provably optimal solutions using branch-and-bound search. Moreover, the exact methodology used reveals a simple renormalization step that improves approximate solutions obtained by any continuous method. The resulting performance gain of discrete algorithms is demonstrated on real-world benchmark data and in extensive Monte Carlo evaluation trials.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference
Pages915-922
Number of pages8
StatePublished - 2005
Externally publishedYes
Event2005 Annual Conference on Neural Information Processing Systems, NIPS 2005 - Vancouver, BC, Canada
Duration: 5 Dec 20058 Dec 2005

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Conference

Conference2005 Annual Conference on Neural Information Processing Systems, NIPS 2005
Country/TerritoryCanada
CityVancouver, BC
Period5/12/058/12/05

Fingerprint

Dive into the research topics of 'Spectral bounds for sparse PCA: Exact and greedy algorithms'. Together they form a unique fingerprint.

Cite this