TY - JOUR

T1 - Sublinear Time Algorithms for Earth Mover's Distance

AU - Ba, Khanh Do

AU - Nguyen, Huy L.

AU - Nguyen, Huy N.

AU - Rubinfeld, Ronitt

PY - 2011/2

Y1 - 2011/2

N2 - We study the problem of estimating the Earth Mover's Distance (EMD) between probability distributions when given access only to samples of the distribution. We give closeness testers and additive-error estimators over domains in [0,1]d, with sample complexities independent of domain size-permitting the testability even of continuous distributions over infinite domains. Instead, our algorithms depend on the dimension of the domain space and the quality of the result required. We also prove lower bounds for closeness testing, showing the dependencies on these parameters to be essentially optimal. Additionally, we consider whether natural classes of distributions exist for which there are algorithms with better dependence on the dimension, and show that for highly clusterable data, this is indeed the case. Lastly, we consider a variant of the EMD, defined over tree metrics instead of the usual ℓ1 metric, and give tight upper and lower bounds.

AB - We study the problem of estimating the Earth Mover's Distance (EMD) between probability distributions when given access only to samples of the distribution. We give closeness testers and additive-error estimators over domains in [0,1]d, with sample complexities independent of domain size-permitting the testability even of continuous distributions over infinite domains. Instead, our algorithms depend on the dimension of the domain space and the quality of the result required. We also prove lower bounds for closeness testing, showing the dependencies on these parameters to be essentially optimal. Additionally, we consider whether natural classes of distributions exist for which there are algorithms with better dependence on the dimension, and show that for highly clusterable data, this is indeed the case. Lastly, we consider a variant of the EMD, defined over tree metrics instead of the usual ℓ1 metric, and give tight upper and lower bounds.

KW - Earth Mover's Distance

KW - Property testing

KW - Sublinear-time algorithms

UR - http://www.scopus.com/inward/record.url?scp=78751576010&partnerID=8YFLogxK

U2 - 10.1007/s00224-010-9265-8

DO - 10.1007/s00224-010-9265-8

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:78751576010

VL - 48

SP - 428

EP - 442

JO - Theory of Computing Systems

JF - Theory of Computing Systems

SN - 1432-4350

IS - 2

ER -