Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, Uri Stemmer

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ζ-1 log(1/β)) (for generalization error ζ with confidence 1-β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximate-DP and provide a nearly tight upper bound of O(log∗ |X|), which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of O((log∗ |X|)1.5) by Kaplan et al. We also provide matching upper and lower bounds of (2log∗|X|) for the additive error of private quasi-concave optimization (a related and more general problem). Our improvement is achieved via the novel Reorder-Slice-Compute paradigm for private data analysis which we believe will have further applications.

Original languageEnglish
Title of host publicationSTOC 2023 - Proceedings of the 55th Annual ACM Symposium on Theory of Computing
EditorsBarna Saha, Rocco A. Servedio
PublisherAssociation for Computing Machinery
Pages472-482
Number of pages11
ISBN (Electronic)9781450399135
DOIs
StatePublished - 2 Jun 2023
Event55th Annual ACM Symposium on Theory of Computing, STOC 2023 - Orlando, United States
Duration: 20 Jun 202323 Jun 2023

Publication series

NameProceedings of the Annual ACM Symposium on Theory of Computing
ISSN (Print)0737-8017

Conference

Conference55th Annual ACM Symposium on Theory of Computing, STOC 2023
Country/TerritoryUnited States
CityOrlando
Period20/06/2323/06/23

Funding

FundersFunder number
Blavatnik Family Foundation
Israel Science Foundation1871/19, 1595/19

    Keywords

    • PAC learning
    • differential privacy
    • threshold functions

    Fingerprint

    Dive into the research topics of 'Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization'. Together they form a unique fingerprint.

    Cite this