Collaborative Research: CIF: Small: Robust Machine Learning under Sparse Adversarial Attacks

机器学习算法已被证明在许多应用中是成功的，例如检测手写、将语音转换为文本、检测自动驾驶车辆的交通信号或从医疗数据预测患者的诊断。机器学习模型通常被“训练”来执行指定的任务。这种训练是通过向模型提供许多数据样本并使用算法来调整模型参数来完成的，以便其输出在大多数情况下与所提供的训练输出一致。要可靠有效地执行此任务，存在许多挑战。最近的研究表明，对数据点进行微小的更改可能会导致错误检测。因此，使学习模型对这种数据扰动具有鲁棒性至关重要，特别是在自动驾驶等安全关键型应用中。这个项目的目标是实现这一特定类别的数据扰动称为“稀疏攻击”。稀疏攻击场景是指扰动仅发生在数据的几个坐标中，例如图像中的几个像素。尽管它们的重要性和各种现实世界的应用，稀疏攻击还没有从理论的角度进行广泛的研究。该项目的目标是开发一个理论框架，用于在存在L0范数有界的对抗扰动或所谓的稀疏攻击的情况下进行鲁棒机器学习。关于非稀疏对抗性攻击已经有了重要的理论研究，但对于稀疏环境缺乏这种基本的理解。这部分是由于L0设置中的挑战，即，L0球是非凸的且高度非光滑的。这个项目的第一个目标是研究程式化数学模型的鲁棒分类的基本限制。这将通过提出可证明对L0攻击具有鲁棒性的防御方法以及证明匡威结果来实现。理想情况下，我们的目标是建立严格的可扩展性和匡威界渐近充分表征最佳的鲁棒分类器。出于实际考虑，还将研究所提出的防御方法在其他情况下的性能。特别是，这个项目探讨了建议的鲁棒假设类的泛化特性，以研究有限样本的数据分布未知时的效果。此外，该项目还包括一个评估计划，以实施开发的防御机制，并分析其在学习模型方面的性能，该模型对稀疏攻击具有鲁棒性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估而被认为值得支持。

Machine-learning algorithms have proved successful in many applications, such as detecting handwriting, converting speech to text, detecting traffic signals for autonomous vehicles, or predicting a patient's diagnosis from medical data. A machine-learning model is usually "trained" to perform the designated task. This training is done by feeding many data samples to the model and using algorithms to adjust the model parameters so that its output is consistent with the provided training output most of the time. There are many challenges to performing this task reliably and efficiently. Recent research has shown that making small changes to the data points can lead to misdetection. Therefore, it is critical to make learning models robust against such data perturbations, especially in safety-critical applications such as autonomous driving. This project aims to achieve this for a specific category of data perturbations called "sparse attacks." Sparse-attack scenarios are those in which perturbations occur in only a few coordinates of the data, such as a few pixels in an image. Despite their importance and various real-world applications, sparse attacks have not been widely studied from a theoretical perspective. The goal of this project is to develop a theoretical framework for robust machine learning in the presence of adversarial perturbations that are bounded in L0 norm, or so-called sparse attacks. There have been significant theoretical studies on non-sparse adversarial attacks, but such fundamental understanding has been lacking for the sparse setting. This is partly due to the challenges in the L0 setting, namely, the L0 ball being non-convex and highly non-smooth. The first goal of this project is to study the fundamental limits of robust classification for stylized mathematical models. This will be done by proposing defense methods that are provably robust against L0 attacks, as well as proving converse results. Ideally, one aims to establish tight achievability and converse bounds asymptotically to fully characterize the optimal robust classifier. Motivated by practical considerations, the performance of the proposed defense methods in other scenarios will also be studied. In particular, this project explores the generalization properties of the proposed robust hypothesis class in order to study the effect of finite samples when the data distribution is unknown. Furthermore, the project consists of an evaluation plan to implement the developed defense mechanisms and analyze its performance in terms of learning a model which is robust against sparse attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.