Adversarially Robust Generalization Just Requires More Unlabeled Data (preprint)
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John E. Hopcroft, Liwei Wang
arXiv: 1906.00555   Code
Previous works show that significantly more labeled data is required to achieve adversarially robust generalization. In this paper, we show that just more unlabeled data is required. The key insight is based on a risk decomposition theorem, in which the expected robust risk is separated into two parts: the stability part which measures the prediction stability in the presence of perturbations, and the accuracy part which evaluates the standard classification accuracy. As the stability part does not depend on any label information, we can optimize this part using unlabeled data. Inspired by the theoretical findings, we further show that a practical adversarial training algorithm that leverages unlabeled data can improve adversarial robust generalization on MNIST and Cifar-10.
Core Idea
Tianle Cai
Chen Dan
Di He
Huan Zhang
NeurIPS 2019:
ICLR 2020: