SaTC: CORE: Medium: Collaborative: Towards Robust Machine Learning Systems

机器学习技术，特别是深度神经网络，越来越多地集成到安全和安全关键应用中，如自动驾驶、精准医疗、入侵检测、恶意软件检测和垃圾邮件过滤。许多研究表明，这些模型容易受到对抗性规避攻击的攻击，攻击者会对正常示例进行微小的、精心设计的更改，以欺骗模型做出错误的决策。该项目的目标是通过描述对抗性和非对抗性示例之间的关系，开发利用这种关系来支持更好地检测对抗性示例的机制，以及证明机器学习模型对它们的鲁棒性的指标和方法，来开发对这些漏洞的正式理解和防御。这些理论、算法和指标将共同提高机器学习系统的稳健性，使它们能够更安全地部署在关键任务应用程序中。该团队还将公开他们的数据集和源代码，并在他们自己的课程和研究中与研究生和本科生一起使用它们，特别努力包括来自科学，技术，工程和数学等代表性不足的群体的学生。这项工作还将支持高中外展项目和夏令营，以吸引年轻学生学习机器学习、安全和计算机科学。该项目围绕三个主要目标进行组织，它们结合起来提供了一种全面的方法来建模和防御逃避攻击。第一个要点旨在通过系统的测量研究来描述正常和对抗的例子。这包括考虑特定示例周围不同类型的区域（例如，公制球、流形和转换诱导区域），然后基于结合附近区域其他示例分类的许多算法来表征示例的脆弱性。第二个重点是通过使用区域中的代表性数据点，聚合多个数据点，并使用不同的分类器集来减少使用单个数据点或算法引起的脆弱性，设计针对对抗性示例的稳健防御。第三个要点包括定义建模健壮性的指标，以及利用这些指标来分析模型健壮性的理论和算法。这些包括公制球中对抗性扰动的下界，基于计算成本的鲁棒性指标，相对于训练数据的新数据集的代表性分析，以及利用人类对对抗性的估计的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning techniques, particularly deep neural networks, are increasingly integrated into safety and security-critical applications such as autonomous driving, precision health care, intrusion detection, malware detection, and spam filtering. A number of studies have shown that these models can be vulnerable to adversarial evasion attacks where the attacker makes small, carefully crafted changes to normal examples in order to trick the model into making incorrect decisions. This project's goal is to develop formal understandings of and defenses against these vulnerabilities through characterizing the relationship between adversarial and non-adversarial examples, developing mechanisms that exploit this relationship to support better detection of adversarial examples, and metrics and methods to demonstrate the robustness of machine learning models against them. Together, the theories, algorithms, and metrics developed will improve the robustness of machine learning systems, allowing them to be deployed more securely in mission-critical applications. The team will also make their datasets and source code publicly available and use them in their own courses and research with both graduate and undergraduate students, with particular efforts to include students from underrepresented groups in Science, Technology, Engineering and Math. The work will also support high school outreach programs and summer camps to attract younger students to study machine learning, security, and computer science.The project is organized around three main thrusts that combine to provide a holistic approach to modeling and defending against evasion attacks. The first thrust aims to characterize both normal and adversarial examples via systematic measurement studies. This includes considering different types of regions around specific examples (e.g., metric ball, manifold, and transformation-induced regions) and then characterizing the examples' vulnerability based on a number of algorithms for combining classifications of other examples in the nearby regions. The second thrust focuses on designing robust defenses against adversarial examples by using representative data points in a region, aggregating multiple data points, and using a diverse set of classifiers to reduce the vulnerability induced by using single data points or algorithms. The third thrust involves defining metrics for modeling robustness along with theories and algorithms that leverage those metrics to analyze model robustness. These include lower bounds of adversarial perturbation in metric balls, robustness metrics based on computational costs, analyses of the representativeness of new datasets relative to training data, and methods for leveraging human estimation of adversarialness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.