TY - JOUR
T1 - Sublinear Time Algorithms for Earth Mover's Distance
AU - Ba, Khanh Do
AU - Nguyen, Huy L.
AU - Nguyen, Huy N.
AU - Rubinfeld, Ronitt
PY - 2011/2
Y1 - 2011/2
N2 - We study the problem of estimating the Earth Mover's Distance (EMD) between probability distributions when given access only to samples of the distribution. We give closeness testers and additive-error estimators over domains in [0,1]d, with sample complexities independent of domain size-permitting the testability even of continuous distributions over infinite domains. Instead, our algorithms depend on the dimension of the domain space and the quality of the result required. We also prove lower bounds for closeness testing, showing the dependencies on these parameters to be essentially optimal. Additionally, we consider whether natural classes of distributions exist for which there are algorithms with better dependence on the dimension, and show that for highly clusterable data, this is indeed the case. Lastly, we consider a variant of the EMD, defined over tree metrics instead of the usual ℓ1 metric, and give tight upper and lower bounds.
AB - We study the problem of estimating the Earth Mover's Distance (EMD) between probability distributions when given access only to samples of the distribution. We give closeness testers and additive-error estimators over domains in [0,1]d, with sample complexities independent of domain size-permitting the testability even of continuous distributions over infinite domains. Instead, our algorithms depend on the dimension of the domain space and the quality of the result required. We also prove lower bounds for closeness testing, showing the dependencies on these parameters to be essentially optimal. Additionally, we consider whether natural classes of distributions exist for which there are algorithms with better dependence on the dimension, and show that for highly clusterable data, this is indeed the case. Lastly, we consider a variant of the EMD, defined over tree metrics instead of the usual ℓ1 metric, and give tight upper and lower bounds.
KW - Earth Mover's Distance
KW - Property testing
KW - Sublinear-time algorithms
UR - http://www.scopus.com/inward/record.url?scp=78751576010&partnerID=8YFLogxK
U2 - 10.1007/s00224-010-9265-8
DO - 10.1007/s00224-010-9265-8
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:78751576010
SN - 1432-4350
VL - 48
SP - 428
EP - 442
JO - Theory of Computing Systems
JF - Theory of Computing Systems
IS - 2
ER -