Collaborative Research: CIF: Small: Robust Machine Learning under Sparse Adversarial Attacks

事实证明，机器学习算法在许多应用中都是成功的，比如检测笔迹、将语音转换为文本、检测自动驾驶汽车的交通信号，或者根据医疗数据预测病人的诊断。机器学习模型通常被“训练”以执行指定的任务。这种训练是通过向模型提供许多数据样本，并使用算法来调整模型参数，使其输出在大多数情况下与提供的训练输出一致来完成的。要可靠而有效地执行这项任务，有许多挑战。最近的研究表明，对数据点进行微小的改变可能会导致误判。因此，使学习模型对此类数据扰动具有鲁棒性至关重要，特别是在自动驾驶等安全关键应用中。这个项目旨在为一种被称为“稀疏攻击”的特定类型的数据扰动实现这一目标。稀疏攻击场景是指仅在数据的几个坐标上发生扰动，例如图像中的几个像素。尽管稀疏攻击很重要，并且在现实世界中有各种各样的应用，但从理论的角度来看，稀疏攻击还没有得到广泛的研究。这个项目的目标是在L0范数的对抗性扰动或所谓的稀疏攻击存在的情况下，为鲁棒机器学习开发一个理论框架。对于非稀疏对抗性攻击已经有了大量的理论研究，但缺乏对稀疏环境的基本理解。这在一定程度上是由于L0设置的挑战，即L0球是非凸的，高度不光滑。该项目的第一个目标是研究程式化数学模型的鲁棒分类的基本限制。这将通过提出可证明对L0攻击具有鲁棒性的防御方法以及证明相反的结果来实现。理想情况下，人们的目标是渐近地建立紧密可达性和逆界，以充分表征最优鲁棒分类器。出于实际考虑，我们还将研究所提出的防御方法在其他场景下的性能。特别地，本项目探讨了所提出的鲁棒假设类的泛化性质，以便研究数据分布未知时有限样本的影响。此外，该项目还包括一个评估计划，以实现所开发的防御机制，并从学习一个对稀疏攻击具有鲁棒性的模型的角度分析其性能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine-learning algorithms have proved successful in many applications, such as detecting handwriting, converting speech to text, detecting traffic signals for autonomous vehicles, or predicting a patient's diagnosis from medical data. A machine-learning model is usually "trained" to perform the designated task. This training is done by feeding many data samples to the model and using algorithms to adjust the model parameters so that its output is consistent with the provided training output most of the time. There are many challenges to performing this task reliably and efficiently. Recent research has shown that making small changes to the data points can lead to misdetection. Therefore, it is critical to make learning models robust against such data perturbations, especially in safety-critical applications such as autonomous driving. This project aims to achieve this for a specific category of data perturbations called "sparse attacks." Sparse-attack scenarios are those in which perturbations occur in only a few coordinates of the data, such as a few pixels in an image. Despite their importance and various real-world applications, sparse attacks have not been widely studied from a theoretical perspective. The goal of this project is to develop a theoretical framework for robust machine learning in the presence of adversarial perturbations that are bounded in L0 norm, or so-called sparse attacks. There have been significant theoretical studies on non-sparse adversarial attacks, but such fundamental understanding has been lacking for the sparse setting. This is partly due to the challenges in the L0 setting, namely, the L0 ball being non-convex and highly non-smooth. The first goal of this project is to study the fundamental limits of robust classification for stylized mathematical models. This will be done by proposing defense methods that are provably robust against L0 attacks, as well as proving converse results. Ideally, one aims to establish tight achievability and converse bounds asymptotically to fully characterize the optimal robust classifier. Motivated by practical considerations, the performance of the proposed defense methods in other scenarios will also be studied. In particular, this project explores the generalization properties of the proposed robust hypothesis class in order to study the effect of finite samples when the data distribution is unknown. Furthermore, the project consists of an evaluation plan to implement the developed defense mechanisms and analyze its performance in terms of learning a model which is robust against sparse attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.