CRII: SaTC: Backdoor Detection, Mitigation, and Prevention in Deep Neural Networks

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。从Alexa到自动驾驶汽车，深度学习到机器学习的方法正在迅速改变我们的工作和生活方式，在从集中式服务器到完全分布式物联网（IoT）配置的环境中变得普遍和普遍。这种普遍性使得基于深度学习的系统越来越成为各种网络攻击的目标，例如生成对抗性示例，欺骗深度学习分类器做出错误的决策。研究较少的攻击是攻击者能够破坏现有模型的训练，或者分发他们创建的模型（例如，作为软件库的一部分），其中包含“后门”，攻击者可以创建系统输入的方式，授予未经授权的访问或导致可预测的错误或失败。这个项目的目标是列出神经网络后门创建的基本原则、理论和限制，以及检测和减轻它们的技术和测试平台。除了这个特定的项目之外，该测试平台将围绕神经网络进行各种各样的研究问题，这项工作还将为K-12教师、学生和家长提供有关安全的培训和教育机会。该项目的一个关键推力是系统地调查现有的神经后门攻击，以了解基本和可推广的攻击原理。基于这些发现，研究团队将(1)设计算法以准确检测嵌入深度学习模型中的神经后门；(2)开发鲁棒的后门根除方案以保证模型恢复；(3)研究预防性防御措施，使其在训练过程中更难形成后门。与上述研究任务并行，研究者将开发一个神经后门测试平台的开发和实验环境，收集神经后门库和数据集，目标是支持围绕神经后门的标准化、可复制研究，最终更广泛地支持神经网络。总的来说，拟议的工作将导致使技术能够保护深度学习系统，加速其发展并扩大其在各种应用领域的可信赖采用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).From Alexa to self-driving vehicles, deep learning approaches to machine learning are rapidly transforming how we work and live, becoming prevalent and pervasive in contexts from centralized servers to fully distributed Internet-of-Things (IoT) configurations. This ubiquity makes deep learning-based systems an increasingly attractive target for a variety of cyberattacks, such as the generation of adversarial examples that trick a deep learning classifier into making incorrect decisions. Less studied are attacks in which attackers are able to corrupt the training of an existing model, or distribute a model they created (for instance, as part of a software library) that contains “backdoors”, ways that an attacker can create system inputs that grant unwarranted access or lead to predictable errors or failures. This project’s goal is to lay out the fundamental principles, theories, and constraints on the creation of neural network backdoors, along with techniques and testbeds for detecting and mitigating them. The testbed will enable a wide variety of research questions around neural networks beyond this specific project, and the work will also provide training and education opportunities around security for K-12 teachers, students, and parents. A key thrust of the project is to systematically investigate existing neural backdoor attacks to understand fundamental and generalizable attack principles. Based on those findings, the research team will (1) devise algorithms to accurately detect neural backdoors embedded in deep learning models, (2) develop robust backdoor eradication schemes for guaranteed model recovery, and (3) investigate preventive defense measure to make it harder to form backdoors during the training process. In parallel to the above research tasks, the investigator will develop a Development and Experimental Environment for Neural Backdoor testbed that collects neural backdoor libraries and datasets, with the goal of supporting standardized, replicable research around neural backdoors and eventually neural networks more generally. Overall, the proposed work will lead to enabling technologies to secure deep learning systems, accelerating their development and widening their trustworthy adoption in various application domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.