TY - GEN

T1 - Testing k-wise and almost k-wise independence

AU - Alon, Noga

AU - Andoni, Alexandr

AU - Kaufman, Tali

AU - Matulef, Kevin

AU - Rubinfeld, Ronitt

AU - Xie, Ning

PY - 2007

Y1 - 2007

N2 - In this work, we consider the problems of testing whether adistribution over (0,1 n) is k-wise (resp. (,k)-wise) independentusing samples drawn from that distribution. For the problem of distinguishing k-wise independent distributions from those that are -far from k-wise independence in statistical distance, we upper bound the number ofrequired samples by (n k/ 2) and lower bound it by (n k-1/2/) (these bounds hold for constantk, and essentially the same bounds hold for general k). Toachieve these bounds, we use Fourier analysis to relate adistribution's distance from k-wise independence to its biases, a measure of the parity imbalance it induces on a setof variables. The relationships we derive are tighter than previouslyknown, and may be of independent interest. To distinguish (,k)-wise independent distributions from thosethat are -far from (,k)-wise independence in statistical distance, we upper bound thenumber of required samples by O(k log n / 22) and lower bound it by ( k log n / 2 k(+) log 1/2 k(+)). Although these bounds are anexponential improvement (in terms of n and k) over thecorresponding bounds for testing k-wise independence, we give evidence thatthe time complexity of testing (,k)-wise independence isunlikely to be poly(n,1/,1/) for k=(log n),since this would disprove a plausible conjecture concerning the hardness offinding hidden cliques in random graphs. Under the conjecture, ourresult implies that for, say, k = log n and = 1 / n 0.99,there is a set of (,k)-wise independent distributions, and a set of distributions at distance =1/n 0.51 from (,k)-wiseindependence, which are indistinguishable by polynomial time algorithms.

AB - In this work, we consider the problems of testing whether adistribution over (0,1 n) is k-wise (resp. (,k)-wise) independentusing samples drawn from that distribution. For the problem of distinguishing k-wise independent distributions from those that are -far from k-wise independence in statistical distance, we upper bound the number ofrequired samples by (n k/ 2) and lower bound it by (n k-1/2/) (these bounds hold for constantk, and essentially the same bounds hold for general k). Toachieve these bounds, we use Fourier analysis to relate adistribution's distance from k-wise independence to its biases, a measure of the parity imbalance it induces on a setof variables. The relationships we derive are tighter than previouslyknown, and may be of independent interest. To distinguish (,k)-wise independent distributions from thosethat are -far from (,k)-wise independence in statistical distance, we upper bound thenumber of required samples by O(k log n / 22) and lower bound it by ( k log n / 2 k(+) log 1/2 k(+)). Although these bounds are anexponential improvement (in terms of n and k) over thecorresponding bounds for testing k-wise independence, we give evidence thatthe time complexity of testing (,k)-wise independence isunlikely to be poly(n,1/,1/) for k=(log n),since this would disprove a plausible conjecture concerning the hardness offinding hidden cliques in random graphs. Under the conjecture, ourresult implies that for, say, k = log n and = 1 / n 0.99,there is a set of (,k)-wise independent distributions, and a set of distributions at distance =1/n 0.51 from (,k)-wiseindependence, which are indistinguishable by polynomial time algorithms.

KW - Almost k-wise independence

KW - Fourier analysis

KW - Hidden clique

KW - K-wise independence

KW - Property testing

UR - http://www.scopus.com/inward/record.url?scp=35448996909&partnerID=8YFLogxK

U2 - 10.1145/1250790.1250863

DO - 10.1145/1250790.1250863

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:35448996909

SN - 1595936319

SN - 9781595936318

T3 - Proceedings of the Annual ACM Symposium on Theory of Computing

SP - 496

EP - 505

BT - STOC'07

T2 - STOC'07: 39th Annual ACM Symposium on Theory of Computing

Y2 - 11 June 2007 through 13 June 2007

ER -