AF: Small: Robust and Secure Learning

机器学习(ML)系统在社会中扮演着越来越重要的角色--从无处不在的语音识别系统，到导航系统、产品推荐系统，以及在制造业、工业和医疗保健领域部署的学习系统。在不久的将来，随着复杂的计算机视觉系统、自动驾驶汽车和ML驱动的医疗保健和患者监控的出现，ML系统有望在我们的社会中几乎无处不在。尽管在理想化的设置中性能很有希望，但当前的ML系统往往很脆弱--它们对输入数据的微小变化很敏感，而且经常存在易被恶意攻击者利用的弱点。解决目前的这些缺点是确保一个严重依赖机器学习的社会的稳定、安全和保障的必要步骤。这个项目的中心目标是开发健壮和安全的学习算法。它们超越了开发实现高精度的学习算法的传统目标，并满足了关键部署系统中对可靠性和安全性的广泛需求。作为该项目研究部分的延伸，调查员将继续开展教育和外联工作。这些措施包括传播该项目产生的研究出版物和代码，继续开发关于以数据为中心的算法、机器学习和相关主题的新课程和教材，以及组织半年一次的工业界和学术界思想交流论坛。该项目的研究核心是解决当前学习和优化算法缺乏健壮性的问题。这种健壮性的缺乏表现为以下两种截然不同的形式。首先，当前的算法对它们所训练的数据集的一小部分变化都很敏感。其次，即使在合法数据上进行训练时，学习的模型往往容易受到“对抗性例子”的影响，因为对于绝大多数数据点--甚至训练集中的数据点--所涉数据点的微小对抗性扰动将导致模型输出一个完全不同的标签。当前学习系统中这两种类型的脆弱性的存在增加了对两种新的安全威胁的脆弱性的可能性：1)威胁，即部分训练数据要么是非常有偏见和不可靠的，要么是更糟糕的--它是由目标是误导机器学习系统的对手生成的；2)部署的机器学习系统的威胁可以通过在其测试点中微小但仔细地生成的对抗性修改来欺骗--修改基本上是人类看不见的。该项目试图通过以下方式解决当前系统的这两个关键弱点：1)开发对包括对抗性数据在内的大量任意数据的存在具有健壮性的新算法，这些算法可以应用于许多基本估计、机器学习和优化任务；2)对为什么某些训练算法生成天生容易受到对抗性示例攻击的模型的原因进行严格的理解，并开发工具来减少这种漏洞。此外，这个项目调查稳健和安全学习的计算和信息理论方面，包括发展对任何潜在的权衡的理解，例如训练数据量和计算时间之间的权衡，以及由此产生的训练模型的稳健性或安全性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) systems play an increasingly central role in society--from ubiquitous speech recognition systems, to navigation systems, product recommendation systems, and deployed learning systems across manufacturing, industry, and healthcare. The near future, with complex computer vision systems, self-driving cars, and ML driven medical care and patient monitoring, promises a nearly pervasive presence of ML systems in our society. Despite promising performance in idealized settings, current ML systems are often brittle--they are sensitive to slight changes in the input data, and often have weaknesses that can be easily exploited by a malicious adversary. Resolving these current shortcomings is a necessary step in ensuring the stability, safety, and security of a society that relies heavily on machine learning. The central goal of this project is to develop learning algorithms that are robust, and secure. These go beyond the traditional goal of developing learning algorithms that achieve high accuracy, and address the broad need for reliability and safety in critical deployed systems. As an extension of the research component of the project, the investigator will continue education and outreach efforts. These include disseminating the research publications and code produced by this project, continuing to develop new courses and teaching materials on data-centric algorithms, machine learning, and related topics, and organizing a semi-annual forum for the exchange of ideas between industry and academia. The research core of this project addresses the lack of robustness of current learning and optimization algorithms. This lack of robustness takes the following two distinct forms. First, current algorithms are sensitive to changes in even a very small portion of the data-set on which they are trained. Second, even when trained on legitimate data, the learned models are often susceptible to "adversarial examples" in the sense that for the vast majority of data points--even data points in the training set--a small adversarial perturbation of the data point in question will result in the model outputting a completely different label. The presence of these two types of fragility in current learning systems raises the possibility of vulnerabilities to two new sorts of security threats: 1) the threat that a portion of the training data is either extremely biased and unreliable, or worse--that it has been generated by an adversary whose goal is to mislead the machine learning system, and 2) the threat that deployed machine learning systems can be tricked via minute but carefully generated adversarial modifications in their test points--modifications that are essentially invisible to humans. The project seeks to address these two critical weaknesses of current systems, by : 1) developing new algorithms that are robust to the presence of significant fractions of arbitrary -- including adversarial -- data, which can be applied to a number of fundamental estimation, machine learning, and optimization tasks, and 2) developing a rigorous understanding of why certain training algorithms yield models that are inherently vulnerable to adversarial examples, and develop tools for reducing this vulnerability. Additionally, this project investigates the computational, and information theoretic aspects of robust and secure learning, including developing an understanding of any potential trade-offs, for example between the amount of training data and computation time, and robustness or security of the resulting trained model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.