Collaborative Research: CIF: Small: Robust Machine Learning under Sparse Adversarial Attacks

机器学习算法已被证明在许多应用中取得了成功，例如检测笔迹、将语音转换为文本、检测自动驾驶车辆的交通信号，或者根据医疗数据预测患者的诊断。机器学习模型通常经过“训练”来执行指定的任务。这种训练是通过向模型提供大量数据样本并使用算法调整模型参数来完成的，以使其输出在大部分时间内与所提供的训练输出一致。要可靠、高效地完成这项任务，还有许多挑战。最近的研究表明，对数据点进行微小的更改可能会导致误判。因此，使学习模型对这种数据扰动具有健壮性是至关重要的，特别是在自动驾驶等安全关键应用中。该项目旨在针对一种称为“稀疏攻击”的特定类别的数据扰动实现这一点。稀疏攻击场景是仅在数据的几个坐标中发生扰动的场景，例如图像中的几个像素。尽管稀疏攻击很重要，在现实世界中也有各种各样的应用，但从理论角度来看，稀疏攻击还没有得到广泛的研究。这个项目的目标是开发一个理论框架，用于在存在L0范数有界的对抗性扰动或所谓的稀疏攻击的情况下进行稳健的机器学习。关于非稀疏对抗性攻击的理论研究已有很多，但对稀疏背景下的对抗性攻击缺乏基本的认识。这在一定程度上是由于L0设置中的挑战，即L0球是非凸和高度非光滑的。这个项目的第一个目标是研究样式化数学模型的稳健分类的基本极限。这将通过提出可证明对L0攻击具有健壮性的防御方法来实现，并证明相反的结果。理想情况下，我们的目标是建立严格的可实现性和渐近性的可逆界，以充分刻画最优的稳健分类器。出于实际考虑，还将研究所提出的防御方法在其他场景中的性能。特别是，本项目探索了所提出的稳健假设类的泛化性质，以便研究在数据分布未知的情况下有限样本的影响。此外，该项目包括一个评估计划，以实施开发的防御机制，并从学习一个对稀疏攻击具有健壮性的模型的角度分析其性能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine-learning algorithms have proved successful in many applications, such as detecting handwriting, converting speech to text, detecting traffic signals for autonomous vehicles, or predicting a patient's diagnosis from medical data. A machine-learning model is usually "trained" to perform the designated task. This training is done by feeding many data samples to the model and using algorithms to adjust the model parameters so that its output is consistent with the provided training output most of the time. There are many challenges to performing this task reliably and efficiently. Recent research has shown that making small changes to the data points can lead to misdetection. Therefore, it is critical to make learning models robust against such data perturbations, especially in safety-critical applications such as autonomous driving. This project aims to achieve this for a specific category of data perturbations called "sparse attacks." Sparse-attack scenarios are those in which perturbations occur in only a few coordinates of the data, such as a few pixels in an image. Despite their importance and various real-world applications, sparse attacks have not been widely studied from a theoretical perspective. The goal of this project is to develop a theoretical framework for robust machine learning in the presence of adversarial perturbations that are bounded in L0 norm, or so-called sparse attacks. There have been significant theoretical studies on non-sparse adversarial attacks, but such fundamental understanding has been lacking for the sparse setting. This is partly due to the challenges in the L0 setting, namely, the L0 ball being non-convex and highly non-smooth. The first goal of this project is to study the fundamental limits of robust classification for stylized mathematical models. This will be done by proposing defense methods that are provably robust against L0 attacks, as well as proving converse results. Ideally, one aims to establish tight achievability and converse bounds asymptotically to fully characterize the optimal robust classifier. Motivated by practical considerations, the performance of the proposed defense methods in other scenarios will also be studied. In particular, this project explores the generalization properties of the proposed robust hypothesis class in order to study the effect of finite samples when the data distribution is unknown. Furthermore, the project consists of an evaluation plan to implement the developed defense mechanisms and analyze its performance in terms of learning a model which is robust against sparse attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.