SaTC: CORE: Medium: Collaborative: Towards Robust Machine Learning Systems

机器学习技术，特别是深度神经网络，越来越多地集成到安全和安全关键型应用中，如自动驾驶、精准医疗、入侵检测、恶意软件检测和垃圾邮件过滤。 许多研究表明，这些模型可能容易受到对抗性规避攻击，攻击者对正常示例进行精心设计的微小更改，以欺骗模型做出错误的决定。 该项目的目标是通过描述对抗性和非对抗性示例之间的关系，开发利用这种关系来支持更好地检测对抗性示例的机制，以及证明机器学习模型对它们的鲁棒性的度量和方法，来开发对这些漏洞的正式理解和防御。 这些理论、算法和指标将共同提高机器学习系统的鲁棒性，使它们能够更安全地部署在关键任务应用程序中。 该团队还将公开他们的数据集和源代码，并将其用于自己的课程和研究生和本科生的研究中，特别努力将科学，技术，工程和数学领域代表性不足的学生纳入其中。这项工作还将支持高中外展计划和夏令营，以吸引年轻学生学习机器学习，安全，和计算机科学。该项目是围绕三个主要推力组织的，这些推力联合收割机提供了一个整体的方法来建模和防御规避攻击。 第一个目标是通过系统的测量研究来描述正常和对抗性的例子。 这包括考虑特定示例周围的不同类型的区域（例如，度量球、流形和变换引起的区域），然后基于用于组合附近区域中的其他示例的分类的多个算法来表征示例的脆弱性。 第二个重点是通过使用一个区域中的代表性数据点，聚合多个数据点，并使用不同的分类器集合来减少使用单个数据点或算法引起的漏洞，来设计针对对抗性示例的强大防御。 第三个推力涉及定义建模鲁棒性的度量沿着，以及利用这些度量来分析模型鲁棒性的理论和算法。 其中包括度量球中对抗扰动的下限、基于计算成本的鲁棒性度量、新数据集相对于训练数据的代表性分析以及利用人类对对抗性的估计的方法。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Machine learning techniques, particularly deep neural networks, are increasingly integrated into safety and security-critical applications such as autonomous driving, precision health care, intrusion detection, malware detection, and spam filtering. A number of studies have shown that these models can be vulnerable to adversarial evasion attacks where the attacker makes small, carefully crafted changes to normal examples in order to trick the model into making incorrect decisions. This project's goal is to develop formal understandings of and defenses against these vulnerabilities through characterizing the relationship between adversarial and non-adversarial examples, developing mechanisms that exploit this relationship to support better detection of adversarial examples, and metrics and methods to demonstrate the robustness of machine learning models against them. Together, the theories, algorithms, and metrics developed will improve the robustness of machine learning systems, allowing them to be deployed more securely in mission-critical applications. The team will also make their datasets and source code publicly available and use them in their own courses and research with both graduate and undergraduate students, with particular efforts to include students from underrepresented groups in Science, Technology, Engineering and Math. The work will also support high school outreach programs and summer camps to attract younger students to study machine learning, security, and computer science.The project is organized around three main thrusts that combine to provide a holistic approach to modeling and defending against evasion attacks. The first thrust aims to characterize both normal and adversarial examples via systematic measurement studies. This includes considering different types of regions around specific examples (e.g., metric ball, manifold, and transformation-induced regions) and then characterizing the examples' vulnerability based on a number of algorithms for combining classifications of other examples in the nearby regions. The second thrust focuses on designing robust defenses against adversarial examples by using representative data points in a region, aggregating multiple data points, and using a diverse set of classifiers to reduce the vulnerability induced by using single data points or algorithms. The third thrust involves defining metrics for modeling robustness along with theories and algorithms that leverage those metrics to analyze model robustness. These include lower bounds of adversarial perturbation in metric balls, robustness metrics based on computational costs, analyses of the representativeness of new datasets relative to training data, and methods for leveraging human estimation of adversarialness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.