SBIR Phase I: Post-training deep neural networks certification against backdoor data poisoning attacks

这个小企业创新研究（SBIR）第一阶段项目的更广泛影响将是保护和认证深度学习模型，这些模型在许多安全和对安全敏感的应用中无处不在，如金融、卫生、军事/情报、网络安全、关键基础设施和个人/消费者使用。预计在这些领域中，深度神经网络（DNN）的部署将在短期内出现强劲增长，其中一些领域受到监管要求的约束，即人工智能（AI）模型必须经过认证，才能像宣传的那样运行。本课题提出了一种针对后门攻击的自信认证新方法。这个小企业创新研究第一阶段项目将为dnn提供针对规避后门攻击（BAs）的数学原则认证服务的第一个商业原型。所提出的方法是完全无监督的，不需要已知的中毒dnn的例子，也不需要任何来自（可能中毒的）训练集的样本。该项目将通过利用具有成本效益的云计算服务的并行计算，推进一种广泛适用和计算效率高的方法，以解决诸如非常大的输入特征空间维度和类别数量以及非常大的深度神经网络等挑战。另一个挑战是使检测器对后门模式（BP）被合并到跨不同应用领域的样本中的机制（例如，加法、乘法）不敏感。除了“静态”dnn和图像域，原型将能够：处理循环dnn；处理时间序列、点云和文档数据域；并为用于时间序列预测和回归的人工智能辩护。将开发api来扩展原型以防御相关攻击，例如，可感知但“场景似是而非”的后门模式或测试时间逃避攻击。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this Small Business Innovation Research (SBIR) Phase I project will be to secure and certify deep learning models that are becoming ubiquitous in many safety and security-sensitive applications, such as finance, health, military/intelligence, cyber security, critical infrastructure, and personal/consumer use. Strong growth in deep neural network (DNN) deployments is forecast in the near term in several of these domains, some of which are subject to regulatory requirements that artificial intelligence (AI) models be certified to perform as advertised. This project proposes a new method to confidently certify against backdoor attacks. This Small Business Innovation Research Phase I project will provide the first commercial prototype of a mathematically principled certification service for DNNs against evasive backdoor attacks (BAs). The proposed method is wholly unsupervised, requiring no known examples of poisoned DNNs nor any samples from the (possibly poisoned) training set. This project will advance a broadly applicable and computationally efficient approach through parallel computation leveraging cost-effective cloud-computing services, to address challenges such as very large input feature space dimensions and number of classes, as well as very large DNNs. Another challenge is to make the detector insensitive to the mechanism (e.g., additive, multiplicative) by which the backdoor pattern (BP) is incorporated into a sample across different application domains. In addition to "static" DNNs and image domains, the prototype will be able to: process recurrent DNNs; handle time series, point cloud, and document data domains; and defend AIs used for time-series prediction and regression. APIs will be developed to expand the prototype to defend against related attacks, e.g., backdoor patterns that are perceptible but "scene plausible" or test-time evasion attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.