Nightmare at test time: Robust learning by feature deletion

Amir Globerson*, Sam Roweis

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


When constructing a classifier from labeled data, it is important not to assign too much weight to any single input feature, in order to increase the robustness of the classifier. This is particularly important in domains with nonstationary feature distributions or with input sensor failures. A common approach to achieving such robustness is to introduce regularization which spreads the weight more evenly between the features. However, this strategy is very generic, and cannot induce robustness specifically tailored to the classification task at hand. In this work, we introduce a new algorithm for avoiding single feature over-weighting by analyzing robustness using a game theoretic formalization. We develop classifiers which are optimally resilient to deletion of features in a minimax sense, and show how to construct such classifiers using quadratic programming. We illustrate the applicability of our methods on spam filtering and handwritten digit recognition tasks, where feature deletion is indeed a realistic noise model.

Original languageEnglish
Title of host publicationACM International Conference Proceeding Series - Proceedings of the 23rd International Conference on Machine Learning, ICML 2006
Number of pages8
StatePublished - 2006
Externally publishedYes
Event23rd International Conference on Machine Learning, ICML 2006 - Pittsburgh, PA, United States
Duration: 25 Jun 200629 Jun 2006

Publication series

NameACM International Conference Proceeding Series


Conference23rd International Conference on Machine Learning, ICML 2006
Country/TerritoryUnited States
CityPittsburgh, PA


Dive into the research topics of 'Nightmare at test time: Robust learning by feature deletion'. Together they form a unique fingerprint.

Cite this