Understanding Simultaneous Train and Test Robustness
Understanding Simultaneous Train and Test Robustness
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Pranjal Awasthi;Sivaraman Balakrishnan;Aravindan Vijayaraghavan
中科院分区：
文献类型：
--
作者：
Pranjal Awasthi;Sivaraman Balakrishnan;Aravindan Vijayaraghavan
本文主要研究鲁棒学习算法。在实际环境中，希望实现对数据分布中许多不同类型的损坏和变化的鲁棒性，例如防御对抗性示例，处理协变量变化和训练数据的污染（数据中毒）。虽然最近有大量的工作在这些主题上，这些不同的鲁棒性概念的模型和算法在很大程度上是孤立开发的。在本文中，我们提出了一个自然的鲁棒性概念，允许我们同时推理训练时间和测试时间的损坏，可以使用各种距离度量（例如，总变异距离（Wasserstein distance）。我们在监督和无监督学习的三个基本设置中研究我们提出的概念（回归，分类和均值估计）。在每种情况下，我们都设计了样本和时间有效的学习算法，具有强大的同步训练和测试鲁棒性保证。特别是，我们的工作表明，训练时和测试时两个看似不同的鲁棒性概念是密切相关的，并且可以利用这种联系来开发适用于这两种环境的算法技术。
This work concerns the study of robust learning algorithms. In practical settings, it is desirable to achieve robustness to many different types of corruptions and shifts in the data distribution such as defending against adversarial examples, dealing with covariate shifts, and contamination of training data (data poisoning). While there has been extensive recent work on these topics, models and algorithms for these different notions of robustness have been largely developed in isolation. In this paper, we propose a natural notion of robustness that allows us to simultaneously reason about train-time and test-time corruptions, that can be measured using various distance metrics (e.g., total variation distance, Wasserstein distance). We study our proposed notion in three fundamental settings in supervised and unsupervised learning (of regression, classification and mean estimation). In each case we design sample and time-efficient learning algorithms with strong simultaneous train-and-test robustness guarantees. In particular, our work shows that the two seemingly different notions of robustness at train-time and test-time are closely related, and this connection can be leveraged to develop algorithmic techniques that are applicable in both the settings.