TY - GEN
T1 - Learned Interpolation for Better Streaming Quantile Approximation with Worst-Case Guarantees
AU - Schiefer, Nicholas
AU - Chen, Justin Y.
AU - Indyk, Piotr
AU - Narayanan, Shyam
AU - Silwal, Sandeep
AU - Wagner, Tal
N1 - Publisher Copyright:
© 2023 Copyright for this paper is retained by the authors.
PY - 2023
Y1 - 2023
N2 - An ε-approximate quantile sketch over a stream of n inputs approximates the rank of any query point q—that is, the number of input points less than q—up to an additive error of εn, generally with some probability of at least 1-1/poly(n), while consuming o(n) space. While the celebrated KLL sketch of Karnin, Lang, and Liberty achieves a provably optimal quantile approximation algorithm over worst-case streams, the approximations it achieves in practice are often far from optimal. Indeed, the most commonly used technique in practice is Dunning’s t-digest, which often achieves much better approximations than KLL on real-world data but is known to have arbitrarily large errors in the worst case. We apply interpolation techniques to the streaming quantiles problem to attempt to achieve better approximations on real-world data sets than KLL while maintaining similar guarantees in the worst case.
AB - An ε-approximate quantile sketch over a stream of n inputs approximates the rank of any query point q—that is, the number of input points less than q—up to an additive error of εn, generally with some probability of at least 1-1/poly(n), while consuming o(n) space. While the celebrated KLL sketch of Karnin, Lang, and Liberty achieves a provably optimal quantile approximation algorithm over worst-case streams, the approximations it achieves in practice are often far from optimal. Indeed, the most commonly used technique in practice is Dunning’s t-digest, which often achieves much better approximations than KLL on real-world data but is known to have arbitrarily large errors in the worst case. We apply interpolation techniques to the streaming quantiles problem to attempt to achieve better approximations on real-world data sets than KLL while maintaining similar guarantees in the worst case.
UR - http://www.scopus.com/inward/record.url?scp=85208787487&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85208787487
T3 - SIAM Conference on Applied and Computational Discrete Algorithms, ACDA 2023
SP - 87
EP - 97
BT - SIAM Conference on Applied and Computational Discrete Algorithms, ACDA 2023
A2 - Berry, Jonathan
A2 - Shmoys, David
A2 - Cowen, Lenore
A2 - Naumann, Uwe
PB - Society for Industrial and Applied Mathematics (SIAM)
T2 - 2nd SIAM Conference on Applied and Computational Discrete Algorithms, ACDA 2023
Y2 - 31 May 2023 through 2 June 2023
ER -