TY - JOUR
T1 - Privately Learning Thresholds
T2 - 33rd Conference on Learning Theory, COLT 2020
AU - Kaplan, Haim
AU - Ligett, Katrina
AU - Mansour, Yishay
AU - Naor, Moni
AU - Stemmer, Uri
N1 - Publisher Copyright:
© 2020 H. Kaplan, K. Ligett, Y. Mansour, M. Naor & U. Stemmer.
PY - 2020
Y1 - 2020
N2 - We study the sample complexity of learning threshold functions under the constraint of differential privacy. It is assumed that each labeled example in the training data is the information of one individual and we would like to come up with a generalizing hypothesis h while guaranteeing differential privacy for the individuals. Intuitively, this means that any single labeled example in the training data should not have a significant effect on the choice of the hypothesis. This problem has received much attention recently; unlike the non-private case, where the sample complexity is independent of the domain size and just depends on the desired accuracy and confidence, for private learning the sample complexity must depend on the domain size X (even for approximate differential privacy). Alon et al. (STOC 2019) showed a lower bound of Ω(log∗ |X|) on the sample complexity and Bun et al. (FOCS 2015) presented an approximate-private learner with sample complexity Õ (2log∗ |X|). In this work we reduce this gap significantly, almost settling the sample complexity. We first present a new upper bound (algorithm) of Õ ((log∗ |X|)2) on the sample complexity and then present an improved version with sample complexity Õ ((log∗ |X|)1.5) . Our algorithm is constructed for the related interior point problem, where the goal is to find a point between the largest and smallest input elements. It is based on selecting an input-dependent hash function and using it to embed the database into a domain whose size is reduced logarithmically; this results in a new database, an interior point of which can be used to generate an interior point of the original database in a differentially private manner.
AB - We study the sample complexity of learning threshold functions under the constraint of differential privacy. It is assumed that each labeled example in the training data is the information of one individual and we would like to come up with a generalizing hypothesis h while guaranteeing differential privacy for the individuals. Intuitively, this means that any single labeled example in the training data should not have a significant effect on the choice of the hypothesis. This problem has received much attention recently; unlike the non-private case, where the sample complexity is independent of the domain size and just depends on the desired accuracy and confidence, for private learning the sample complexity must depend on the domain size X (even for approximate differential privacy). Alon et al. (STOC 2019) showed a lower bound of Ω(log∗ |X|) on the sample complexity and Bun et al. (FOCS 2015) presented an approximate-private learner with sample complexity Õ (2log∗ |X|). In this work we reduce this gap significantly, almost settling the sample complexity. We first present a new upper bound (algorithm) of Õ ((log∗ |X|)2) on the sample complexity and then present an improved version with sample complexity Õ ((log∗ |X|)1.5) . Our algorithm is constructed for the related interior point problem, where the goal is to find a point between the largest and smallest input elements. It is based on selecting an input-dependent hash function and using it to embed the database into a domain whose size is reduced logarithmically; this results in a new database, an interior point of which can be used to generate an interior point of the original database in a differentially private manner.
UR - http://www.scopus.com/inward/record.url?scp=85161331799&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???
AN - SCOPUS:85161331799
SN - 2640-3498
VL - 125
SP - 2263
EP - 2285
JO - Proceedings of Machine Learning Research
JF - Proceedings of Machine Learning Research
Y2 - 9 July 2020 through 12 July 2020
ER -