TY - JOUR

T1 - Graph-based discriminators

T2 - 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019

AU - Livni, Roi

AU - Mansour, Yishay

N1 - Publisher Copyright:
© 2019 Neural information processing systems foundation. All rights reserved.

PY - 2019

Y1 - 2019

N2 - A basic question in learning theory is to identify if two distributions are identical when we have access only to examples sampled from the distributions. This basic task is considered, for example, in the context of Generative Adversarial Networks (GANs), where a discriminator is trained to distinguish between a real-life distribution and a synthetic distribution. Classically, we use a hypothesis class H and claim that the two distributions are distinct if for some h ? H the expected value on the two distributions is (significantly) different. Our starting point is the following fundamental problem: "is having the hypothesis dependent on more than a single random example beneficial". To address this challenge we define k-ary based discriminators, which have a family of Boolean k-ary functions G. Each function g ? G naturally defines a hyper-graph, indicating whether a given hyper-edge exists. A function g ? G distinguishes between two distributions, if the expected value of g, on a k-tuple of i.i.d examples, on the two distributions is (significantly) different. We study the expressiveness of families of k-ary functions, compared to the classical hypothesis class H, which is k = 1. We show a separation in expressiveness of k + 1-ary versus k-ary functions. This demonstrate the great benefit of having k = 2 as distinguishers. For k = 2 we introduce a notion similar to the VC-dimension, and show that it controls the sample complexity. We proceed and provide upper and lower bounds as a function of our extended notion of VC-dimension.

AB - A basic question in learning theory is to identify if two distributions are identical when we have access only to examples sampled from the distributions. This basic task is considered, for example, in the context of Generative Adversarial Networks (GANs), where a discriminator is trained to distinguish between a real-life distribution and a synthetic distribution. Classically, we use a hypothesis class H and claim that the two distributions are distinct if for some h ? H the expected value on the two distributions is (significantly) different. Our starting point is the following fundamental problem: "is having the hypothesis dependent on more than a single random example beneficial". To address this challenge we define k-ary based discriminators, which have a family of Boolean k-ary functions G. Each function g ? G naturally defines a hyper-graph, indicating whether a given hyper-edge exists. A function g ? G distinguishes between two distributions, if the expected value of g, on a k-tuple of i.i.d examples, on the two distributions is (significantly) different. We study the expressiveness of families of k-ary functions, compared to the classical hypothesis class H, which is k = 1. We show a separation in expressiveness of k + 1-ary versus k-ary functions. This demonstrate the great benefit of having k = 2 as distinguishers. For k = 2 we introduce a notion similar to the VC-dimension, and show that it controls the sample complexity. We proceed and provide upper and lower bounds as a function of our extended notion of VC-dimension.

UR - http://www.scopus.com/inward/record.url?scp=85090175862&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???

AN - SCOPUS:85090175862

SN - 1049-5258

VL - 32

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

Y2 - 8 December 2019 through 14 December 2019

ER -