Tricking the Hashing Trick: A Tight Lower Bound on the Robustness of CountSketch to Adaptive Inputs

Edith Cohen, Jelani Nelson, Tamás Sarlós, Uri Stemmer

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

CountSketch and Feature Hashing (the “hashing trick”) are popular randomized dimensionality reduction methods that support recovery of ℓ2-heavy hitters (keys i where vi2 > ϵ∥v∥22) and approximate inner products. When the inputs are not adaptive (do not depend on prior outputs), classic estimators applied to a sketch of size O(ℓ/ϵ) are accurate for a number of queries that is exponential in ℓ. When inputs are adaptive, however, an adversarial input can be constructed after O(ℓ) queries with the classic estimator and the best known robust estimator only supports Õ(ℓ2) queries. In this work we show that this quadratic dependence is in a sense inherent: We design an attack that after O(ℓ2) queries produces an adversarial input vector whose sketch is highly biased. Our attack uses “natural” non-adaptive inputs (only the final adversarial input is chosen adaptively) and universally applies with any correct estimator, including one that is unknown to the attacker. In that, we expose inherent vulnerability of this fundamental method.

Original languageEnglish
Title of host publicationAAAI-23 Technical Tracks 6
EditorsBrian Williams, Yiling Chen, Jennifer Neville
PublisherAAAI press
Pages7235-7243
Number of pages9
ISBN (Electronic)9781577358800
StatePublished - 27 Jun 2023
Event37th AAAI Conference on Artificial Intelligence, AAAI 2023 - Washington, United States
Duration: 7 Feb 202314 Feb 2023

Publication series

NameProceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023
Volume37

Conference

Conference37th AAAI Conference on Artificial Intelligence, AAAI 2023
Country/TerritoryUnited States
CityWashington
Period7/02/2314/02/23

Funding

FundersFunder number
Blavatnik Family Foundation
Israel Science Foundation1871/19, 1595/19

    Fingerprint

    Dive into the research topics of 'Tricking the Hashing Trick: A Tight Lower Bound on the Robustness of CountSketch to Adaptive Inputs'. Together they form a unique fingerprint.

    Cite this