TY - GEN

T1 - Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

AU - Cohen, Edith

AU - Lyu, Xin

AU - Nelson, Jelani

AU - Sarlós, Tamás

AU - Stemmer, Uri

N1 - Publisher Copyright:
© 2023 Owner/Author.

PY - 2023/6/2

Y1 - 2023/6/2

N2 - The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ζ-1 log(1/β)) (for generalization error ζ with confidence 1-β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximate-DP and provide a nearly tight upper bound of O(log∗ |X|), which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of O((log∗ |X|)1.5) by Kaplan et al. We also provide matching upper and lower bounds of (2log∗|X|) for the additive error of private quasi-concave optimization (a related and more general problem). Our improvement is achieved via the novel Reorder-Slice-Compute paradigm for private data analysis which we believe will have further applications.

AB - The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ζ-1 log(1/β)) (for generalization error ζ with confidence 1-β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximate-DP and provide a nearly tight upper bound of O(log∗ |X|), which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of O((log∗ |X|)1.5) by Kaplan et al. We also provide matching upper and lower bounds of (2log∗|X|) for the additive error of private quasi-concave optimization (a related and more general problem). Our improvement is achieved via the novel Reorder-Slice-Compute paradigm for private data analysis which we believe will have further applications.

KW - PAC learning

KW - differential privacy

KW - threshold functions

UR - http://www.scopus.com/inward/record.url?scp=85163083598&partnerID=8YFLogxK

U2 - 10.1145/3564246.3585148

DO - 10.1145/3564246.3585148

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85163083598

T3 - Proceedings of the Annual ACM Symposium on Theory of Computing

SP - 472

EP - 482

BT - STOC 2023 - Proceedings of the 55th Annual ACM Symposium on Theory of Computing

A2 - Saha, Barna

A2 - Servedio, Rocco A.

PB - Association for Computing Machinery

T2 - 55th Annual ACM Symposium on Theory of Computing, STOC 2023

Y2 - 20 June 2023 through 23 June 2023

ER -