Prefix Filter: Practically and Theoretically Better Than Bloom

Tomer Even, Guy Even, Adam Morrison

Research output: Contribution to journalConference articlepeer-review

Abstract

Many applications of approximate membership query data structures, or filters, require only an incremental filter that supports insertions but not deletions. However, the design space of incremental filters is missing a “sweet spot” filter that combines space efficiency, fast queries, and fast insertions. Incremental filters, such as the Bloom and blocked Bloom filter, are not space efficient. Dynamic filters (i.e., supporting deletions), such as the cuckoo or vector quotient filter, are space efficient but do not exhibit consistently fast insertions and queries. In this paper, we propose the prefix filter, an incremental filter that addresses the above challenge: (1) its space (in bits) is similar to state-of-the-art dynamic filters; (2) query throughput is high and is comparable to that of the cuckoo filter; and (3) insert throughput is high with overall build times faster than those of the vector quotient filter and cuckoo filter by 1.39×–1.46× and 3.2×–3.5×, respectively. We present a rigorous analysis of the prefix filter that holds also for practical set sizes (i.e., n = 225 ). The analysis deals with the probability of failure, false positive rate, and probability that an operation requires accessing more than a single cache line.

Original languageEnglish
Pages (from-to)1311-1323
Number of pages13
JournalProceedings of the VLDB Endowment
Volume15
Issue number7
DOIs
StatePublished - 2022
Event48th International Conference on Very Large Data Bases, VLDB 2022 - Sydney, Australia
Duration: 5 Sep 20229 Sep 2022

Fingerprint

Dive into the research topics of 'Prefix Filter: Practically and Theoretically Better Than Bloom'. Together they form a unique fingerprint.

Cite this