CRII: SaTC: Backdoor Detection, Mitigation, and Prevention in Deep Neural Networks

该奖项的全部或部分资金来自《2021年美国救援计划法案》(Public Law 117-2)。从Alexa到自动驾驶汽车，机器学习的深度学习方法正在迅速改变我们的工作和生活方式，在从集中式服务器到完全分布式物联网(IoT)配置的环境中变得普遍和普遍。这种普及性使得基于深度学习的系统成为各种网络攻击的越来越有吸引力的目标，例如生成敌意例子，诱骗深度学习分类器做出错误的决定。较少被研究的攻击是攻击者能够破坏现有模型的训练，或分发他们创建的包含“后门”的模型(例如，作为软件库的一部分)，即攻击者创建系统输入的方式，这些输入授予未经授权的访问权限或导致可预测的错误或故障。该项目的目标是展示创建神经网络后门的基本原则、理论和约束，以及检测和缓解这些后门的技术和试验台。试验台将使这个特定项目以外的各种神经网络研究问题成为可能，这项工作还将为K-12教师、学生和家长提供有关安全的培训和教育机会。该项目的一个关键推动力是系统地调查现有的神经后门攻击，以了解基本的和可概括的攻击原理。基于这些发现，研究小组将(1)设计算法来准确检测深度学习模型中嵌入的神经后门，(2)开发健壮的后门消除方案，以保证模型恢复，以及(3)研究预防性防御措施，以使训练过程中更难形成后门。在上述研究任务的同时，研究人员将开发一个神经后门试验台的开发和实验环境，收集神经后门的库和数据集，目标是支持围绕神经后门以及最终更广泛的神经网络的标准化、可复制的研究。总体而言，拟议的工作将导致使技术能够确保深度学习系统的安全，加快它们的开发，并扩大它们在各种应用领域的可信采用。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).From Alexa to self-driving vehicles, deep learning approaches to machine learning are rapidly transforming how we work and live, becoming prevalent and pervasive in contexts from centralized servers to fully distributed Internet-of-Things (IoT) configurations. This ubiquity makes deep learning-based systems an increasingly attractive target for a variety of cyberattacks, such as the generation of adversarial examples that trick a deep learning classifier into making incorrect decisions. Less studied are attacks in which attackers are able to corrupt the training of an existing model, or distribute a model they created (for instance, as part of a software library) that contains “backdoors”, ways that an attacker can create system inputs that grant unwarranted access or lead to predictable errors or failures. This project’s goal is to lay out the fundamental principles, theories, and constraints on the creation of neural network backdoors, along with techniques and testbeds for detecting and mitigating them. The testbed will enable a wide variety of research questions around neural networks beyond this specific project, and the work will also provide training and education opportunities around security for K-12 teachers, students, and parents. A key thrust of the project is to systematically investigate existing neural backdoor attacks to understand fundamental and generalizable attack principles. Based on those findings, the research team will (1) devise algorithms to accurately detect neural backdoors embedded in deep learning models, (2) develop robust backdoor eradication schemes for guaranteed model recovery, and (3) investigate preventive defense measure to make it harder to form backdoors during the training process. In parallel to the above research tasks, the investigator will develop a Development and Experimental Environment for Neural Backdoor testbed that collects neural backdoor libraries and datasets, with the goal of supporting standardized, replicable research around neural backdoors and eventually neural networks more generally. Overall, the proposed work will lead to enabling technologies to secure deep learning systems, accelerating their development and widening their trustworthy adoption in various application domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.