AF: Small: Robust and Secure Learning

机器学习（ML）系统在社会中扮演着越来越重要的角色——从无处不在的语音识别系统，到导航系统、产品推荐系统，以及在制造业、工业和医疗保健领域部署的学习系统。在不久的将来，随着复杂的计算机视觉系统、自动驾驶汽车、机器学习驱动的医疗保健和患者监测，机器学习系统在我们的社会中几乎无处不在。尽管在理想设置中表现良好，但当前的机器学习系统通常很脆弱——它们对输入数据的微小变化很敏感，并且通常存在容易被恶意对手利用的弱点。解决这些当前的缺点是确保一个严重依赖机器学习的社会的稳定、安全和保障的必要步骤。这个项目的中心目标是开发健壮、安全的学习算法。这些都超越了开发高精度学习算法的传统目标，并解决了关键部署系统对可靠性和安全性的广泛需求。作为项目研究部分的延伸，研究者将继续开展教育和推广工作。这些措施包括传播该项目产生的研究出版物和代码，继续开发以数据为中心的算法、机器学习和相关主题的新课程和教材，以及组织半年一次的论坛，以促进产业界和学术界之间的思想交流。该项目的研究核心是解决当前学习和优化算法缺乏鲁棒性的问题。缺乏健壮性表现为以下两种不同的形式。首先，当前的算法对数据集的变化非常敏感，即使是训练它们的数据集的很小一部分。其次，即使在合法数据上进行训练，学习模型也经常容易受到“对抗性示例”的影响，因为对于绝大多数数据点——甚至是训练集中的数据点——数据点的一个小的对抗性扰动将导致模型输出一个完全不同的标签。在当前的学习系统中，这两种脆弱性的存在增加了两种新的安全威胁的脆弱性的可能性：1)部分训练数据要么是极度偏见和不可靠的，要么是更糟糕的威胁——它是由一个目标是误导机器学习系统的对手生成的；2)部署的机器学习系统可以通过在测试点上进行微小但精心生成的对抗性修改来欺骗——这些修改对人类来说基本上是不可见的。该项目力求通过以下方式解决当前系统的这两个关键弱点：1)开发对大量任意数据（包括对抗性数据）具有鲁棒性的新算法，可应用于许多基本估计、机器学习和优化任务；2)对为什么某些训练算法产生的模型本质上容易受到对抗性示例的影响，有一个严格的理解，并开发工具来减少这种脆弱性。此外，该项目还研究了鲁棒和安全学习的计算和信息理论方面，包括开发对任何潜在权衡的理解，例如在训练数据量和计算时间之间，以及最终训练模型的鲁棒性或安全性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) systems play an increasingly central role in society--from ubiquitous speech recognition systems, to navigation systems, product recommendation systems, and deployed learning systems across manufacturing, industry, and healthcare. The near future, with complex computer vision systems, self-driving cars, and ML driven medical care and patient monitoring, promises a nearly pervasive presence of ML systems in our society. Despite promising performance in idealized settings, current ML systems are often brittle--they are sensitive to slight changes in the input data, and often have weaknesses that can be easily exploited by a malicious adversary. Resolving these current shortcomings is a necessary step in ensuring the stability, safety, and security of a society that relies heavily on machine learning. The central goal of this project is to develop learning algorithms that are robust, and secure. These go beyond the traditional goal of developing learning algorithms that achieve high accuracy, and address the broad need for reliability and safety in critical deployed systems. As an extension of the research component of the project, the investigator will continue education and outreach efforts. These include disseminating the research publications and code produced by this project, continuing to develop new courses and teaching materials on data-centric algorithms, machine learning, and related topics, and organizing a semi-annual forum for the exchange of ideas between industry and academia. The research core of this project addresses the lack of robustness of current learning and optimization algorithms. This lack of robustness takes the following two distinct forms. First, current algorithms are sensitive to changes in even a very small portion of the data-set on which they are trained. Second, even when trained on legitimate data, the learned models are often susceptible to "adversarial examples" in the sense that for the vast majority of data points--even data points in the training set--a small adversarial perturbation of the data point in question will result in the model outputting a completely different label. The presence of these two types of fragility in current learning systems raises the possibility of vulnerabilities to two new sorts of security threats: 1) the threat that a portion of the training data is either extremely biased and unreliable, or worse--that it has been generated by an adversary whose goal is to mislead the machine learning system, and 2) the threat that deployed machine learning systems can be tricked via minute but carefully generated adversarial modifications in their test points--modifications that are essentially invisible to humans. The project seeks to address these two critical weaknesses of current systems, by : 1) developing new algorithms that are robust to the presence of significant fractions of arbitrary -- including adversarial -- data, which can be applied to a number of fundamental estimation, machine learning, and optimization tasks, and 2) developing a rigorous understanding of why certain training algorithms yield models that are inherently vulnerable to adversarial examples, and develop tools for reducing this vulnerability. Additionally, this project investigates the computational, and information theoretic aspects of robust and secure learning, including developing an understanding of any potential trade-offs, for example between the amount of training data and computation time, and robustness or security of the resulting trained model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.