SaTC: CORE: Medium: Collaborative: Towards Robust Machine Learning Systems

机器学习技术，特别是深度神经网络，越来越多地集成到安全和安全关键应用中，如自动驾驶、精确医疗保健、入侵检测、恶意软件检测和垃圾邮件过滤。许多研究表明，这些模型容易受到对抗性规避攻击，即攻击者对正常示例进行精心制作的小更改，以便诱骗模型做出错误的决定。该项目的目标是通过表征对抗性和非对抗性示例之间的关系，开发利用这种关系来支持更好地检测对抗性示例的机制，以及展示机器学习模型对它们的健壮性的度量和方法，来对这些漏洞进行正式的理解和防御。总而言之，开发的理论、算法和指标将提高机器学习系统的健壮性，使其能够更安全地部署在任务关键型应用程序中。该团队还将公开他们的数据集和源代码，并在他们自己的课程和研究生和本科生的研究中使用它们，特别是努力纳入科学、技术、工程和数学领域代表性较低的群体的学生。这项工作还将支持高中外展项目和夏令营，以吸引更年轻的学生学习机器学习、安全和计算机科学。该项目围绕三个主要方面组织，结合起来提供了一种全面的建模和防御逃避攻击的方法。第一个主旨旨在通过系统的测量研究来描述正常和敌对的例子。这包括考虑特定示例周围的不同类型的区域(例如，度量球、流形和变换诱导区域)，然后基于用于组合附近区域中的其他示例的分类的多个算法来表征示例的脆弱性。第二个重点是通过使用一个区域中具有代表性的数据点，聚合多个数据点，并使用不同的分类器集来减少使用单个数据点或算法导致的脆弱性，从而设计针对敌意示例的健壮防御。第三个重点涉及定义建模健壮性的指标，以及利用这些指标分析模型健壮性的理论和算法。这些包括公制球中对抗性扰动的下界，基于计算成本的稳健性度量，新数据集相对于训练数据的代表性分析，以及利用人类对对抗性的估计的方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning techniques, particularly deep neural networks, are increasingly integrated into safety and security-critical applications such as autonomous driving, precision health care, intrusion detection, malware detection, and spam filtering. A number of studies have shown that these models can be vulnerable to adversarial evasion attacks where the attacker makes small, carefully crafted changes to normal examples in order to trick the model into making incorrect decisions. This project's goal is to develop formal understandings of and defenses against these vulnerabilities through characterizing the relationship between adversarial and non-adversarial examples, developing mechanisms that exploit this relationship to support better detection of adversarial examples, and metrics and methods to demonstrate the robustness of machine learning models against them. Together, the theories, algorithms, and metrics developed will improve the robustness of machine learning systems, allowing them to be deployed more securely in mission-critical applications. The team will also make their datasets and source code publicly available and use them in their own courses and research with both graduate and undergraduate students, with particular efforts to include students from underrepresented groups in Science, Technology, Engineering and Math. The work will also support high school outreach programs and summer camps to attract younger students to study machine learning, security, and computer science.The project is organized around three main thrusts that combine to provide a holistic approach to modeling and defending against evasion attacks. The first thrust aims to characterize both normal and adversarial examples via systematic measurement studies. This includes considering different types of regions around specific examples (e.g., metric ball, manifold, and transformation-induced regions) and then characterizing the examples' vulnerability based on a number of algorithms for combining classifications of other examples in the nearby regions. The second thrust focuses on designing robust defenses against adversarial examples by using representative data points in a region, aggregating multiple data points, and using a diverse set of classifiers to reduce the vulnerability induced by using single data points or algorithms. The third thrust involves defining metrics for modeling robustness along with theories and algorithms that leverage those metrics to analyze model robustness. These include lower bounds of adversarial perturbation in metric balls, robustness metrics based on computational costs, analyses of the representativeness of new datasets relative to training data, and methods for leveraging human estimation of adversarialness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.