SaTC: CORE: Medium: Collaborative: Using Machine Learning to Build More Resilient and Transparent Computer Systems

机器学习算法越来越多地成为日常生活的一部分：它们帮助我们在浏览网页时看到的广告，现代汽车中的自动驾驶辅助设备，甚至天气预测和关键基础设施。我们依赖这些算法的部分原因是它们的性能优于其他算法，并且它们可以很容易地定制新的应用程序。许多机器学习算法也有一个很大的弱点：很难理解它们如何以及为什么计算它们提供的答案。这种不透明性意味着我们从机器学习算法中得到的答案可能存在微妙的偏见，甚至完全错误，但我们可能没有意识到这一点。该项目的目标是让机器学习算法更容易理解，并利用攻击者使用的一些技术来欺骗机器学习算法，使其出错，以构建更能抵抗攻击的计算机系统。除了对机器学习算法的设计和使用做出基本贡献外，该项目还包括推广工作，吸引学生获得机器学习工具的实践经验。该项目侧重于深度神经网络（DNN）。 在过去的五年里，大量的研究表明，这些模型倾向于被用来愚弄它们的输入所回避--所谓的“对抗性例子”。“这些类型的攻击利用了DNN的不透明性：虽然DNN在某些分类任务上表现得非常好，但它们往往无视对其如何做到这一点的简单解释，而且确实可以利用人类可能会感到惊讶的功能来做到这一点。该项目利用DNN及其攻击来深入了解如何构建更具弹性的计算机系统。具体来说，该项目将使用DNN来模拟试图攻击计算机系统的对手，然后“攻击”这些DNN，以了解如何提高这些系统对攻击的弹性。这种建模将使用生成对抗网络（GAN）完成，其中“生成器”和“代理”模型竞争。这一愿景的核心是在约束条件下规避DNN的能力，以及从DNN中提取关于它们如何执行分类的解释的能力。因此，该项目将在开发欺骗DNN的更好方法和改进这一重要机器学习工具方面取得根本性进展。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning algorithms are increasingly part of everyday life: they help power the ads that we see while browsing the web, self-driving aids in modern cars, and even weather prediction and critical infrastructure. We rely on these algorithms in part because they perform better than alternatives and they can be easy to customize to new applications. Many machine learning algorithms also have a big weakness: it is difficult to understand how and why they compute the answers they provide. This opaqueness means that the answers we get from a machine learning algorithm could be subtly biased or even completely wrong, and yet we might not realize it. This project's goal is to make machine learning algorithms easier to understand, as well as to leverage some of the techniques used by attackers to trick machine learning algorithms into making mistakes to build computer systems that are more resistant to attack. In addition to making fundamental contributions to how machine learning algorithms are designed and used, the project includes outreach efforts that will entice students to gain hands-on experience with machine learning tools.This project focuses on deep neural networks (DNNs). A groundswell of research within the past five years has demonstrated the propensity of these models to being evaded by inputs created to fool them -- so called "adversarial examples." These types of attacks leverage DNNs' opacity: while DNNs can perform remarkably well on some classification tasks, they often defy simple explanations of how they do so, and indeed can leverage features for doing so that humans might find surprising. This project leverages DNNs and the attacks against them to gain insights into how to build more resilient computer systems. Specifically, the project will use DNNs to model adversaries trying to attack computer systems and then "attack" these DNNs to learn how to improve these systems' resilience to attack. This modeling will be done using Generative Adversarial Nets (GANs), in which "generator" and "discriminator" models compete. Central to this vision are the abilities to evade DNNs under constraints and to extract explanations from them about how they perform classification. Consequently, this project will make fundamental advances both in developing better methods to deceive DNNs and in improving this important machine-learning tool.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.