Unlabeled data improves adversarial robustness

Yair Carmon, Aditi Raghunathan*, Ludwig Schmidt, Percy Liang, John C. Duchi

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

324 Scopus citations

Abstract

We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian model of Schmidt et al. [41] that shows a sample complexity gap between standard and robust classification. We prove that unlabeled data bridges this gap: a simple semisupervised learning procedure (self-training) achieves high robust accuracy using the same number of labels required for achieving high standard accuracy. Empirically, we augment CIFAR-10 with 500K unlabeled images sourced from 80 Million Tiny Images and use robust self-training to outperform state-of-the-art robust accuracies by over 5 points in (i) `1 robustness against several strong attacks via adversarial training and (ii) certified `2 and `1 robustness via randomized smoothing. On SVHN, adding the dataset's own extra training set with the labels removed provides gains of 4 to 10 points, within 1 point of the gain from using the extra labels.

Original languageEnglish
JournalAdvances in Neural Information Processing Systems
Volume32
StatePublished - 2019
Externally publishedYes
Event33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 - Vancouver, Canada
Duration: 8 Dec 201914 Dec 2019

Funding

FundersFunder number
National Science Foundation1553086
Alfred P. Sloan FoundationONR-YIP N00014-19-1-2288

    Fingerprint

    Dive into the research topics of 'Unlabeled data improves adversarial robustness'. Together they form a unique fingerprint.

    Cite this