Compressing and Teaching for Low VC-Dimension

Shay Moran, Amir Shpilka, Avi Wigderson, Amir Yehudayoff

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In this work we study the quantitative relation between VC-dimension and two other basic parameters related to learning and teaching. Namely, the quality of sample compression schemes and of teaching sets for classes of low VC-dimension. Let C be a binary concept class of size m and VC-dimension d. Prior to this work, the best known upper bounds for both parameters were log(m), while the best lower bounds are linear in d. We present significantly better upper bounds on both as follows. We construct sample compression schemes of size exp(d) for C. This resolves a question of Littlest one and Warmuth (1986). Roughly speaking, we show that given an arbitrary set of labeled examples from an unknown concept in C, one can retain only a subset of exp(d) of them, in a way that allows to recover the labels of all other examples in the set, using additional exp(d) information bits. We further show that there always exists a concept c in C with a teaching set (i.e. A list of c-labeled examples uniquely identifying c in C) of size exp(d) log log(m). This problem was studied by Kuhlmann (1999). Our construction also implies that the recursive teaching (RT) dimension of C is at most exp(d) log log(m) as well. The RT-dimension was suggested by Zilles et al. And Doliwa et al. (2010). The same notion (under the name partial-ID width) was independently studied by Wigderson and Yehuday off (2013). An upper bound on this parameter that depends only on d is known just for the very simple case d=1, and is open even for d=2. We also make small progress towards this seemingly modest goal.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, FOCS 2015
PublisherIEEE Computer Society
Pages40-51
Number of pages12
ISBN (Electronic)9781467381918
DOIs
StatePublished - 11 Dec 2015
Event56th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2015 - Berkeley, United States
Duration: 17 Oct 201520 Oct 2015

Publication series

NameProceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS
Volume2015-December
ISSN (Print)0272-5428

Conference

Conference56th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2015
Country/TerritoryUnited States
CityBerkeley
Period17/10/1520/10/15

Keywords

  • PAC learning
  • VC dimension
  • recursive teaching dimension
  • sample compression schemes

Fingerprint

Dive into the research topics of 'Compressing and Teaching for Low VC-Dimension'. Together they form a unique fingerprint.

Cite this