SaTC: CORE: Medium: Digital Forensics for Deep Neural Networks

深度神经网络（dnn）在对抗性攻击面前的脆弱性仍然是限制其在自动驾驶汽车、医疗诊断和金融市场分析等安全关键应用中广泛应用的主要障碍。围绕对抗性健壮性的现有研究已经开发了一系列有希望的防御，但这些防御通常在此后不久就被更强的适应性攻击所打破。这似乎不太可能改变，需要补充安全工作，增加成功攻击的成本（时间和计算资源），减少攻击的影响，并在攻击后识别攻击者和漏洞。该项目的最终目标是通过开发攻击取证方法来提高深度学习系统的鲁棒性。通过将防御工具集从攻击预防扩展到包括攻击后分析，该项目可以改变攻击者和防御者之间的竞争环境。该项目将考虑攻击dnn的三种主要途径。第一个重点是分析训练时间中毒攻击，使用聚类和机器去学习启发的方法来识别中毒样本，并开发模型训练方法来跟踪训练来源，以支持后来的取证。第二个重点是在使用模型时的推理时间攻击，通过开发包含额外数据的目标模型的变体；选择额外的数据是为了保持准确性，同时允许防御者推断用于生成攻击的模型。第三部分侧重于识别用于生成图像、文本和其他内容的神经合成模型产生的新漏洞，描述攻击者可能拥有的目标空间以及他们可能用来毒害这些模型的方法。项目团队还将与行业合作伙伴合作，为这项工作奠定基础，增加其转化为实际系统的机会，并与他们的部门合作，增加网络安全研究人员的数量和多样性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The vulnerability of Deep Neural Networks (DNNs) to adversarial attacks remains a major hurdle limiting their use in a wide range of safety-critical applications, such as self-driving cars, medical diagnosis, and financial market analysis. Existing research around adversarial robustness has developed a series of promising defenses, but these are generally broken shortly thereafter by stronger adaptive attacks. This seems unlikely to change, calling for complementary security efforts that increase the cost (in time and computing resources) for successful attacks, reduce the impact of attacks, and identify attackers and vulnerabilities following an attack. This project targets the final goal through developing attack forensics methods to improve the robustness of deep learning systems. The project can alter the playing field between attackers and defenders, by expanding the defense toolset beyond attack prevention to include post-attack analysis.The project will consider three main avenues of attacks on DNNs. The first focuses on analyzing training time poisoning attacks, using clustering and machine unlearning-inspired methods to identify poisoned samples and developing model training methods that trace training provenance to support later forensics. The second focuses on inference time attacks while models are in use, through developing variations of target models that include additional data; the additional data is chosen to maintain accuracy while allowing defenders to make inferences about the models being used to generate attacks. The third focuses on identifying new vulnerabilities arising from neural synthesis models used to generate images, text, and other content, characterizing the space of possible goals attackers might have and methods they might use to poison these models. The project team will also work with industry partners to ground the work and increase the chance of it transferring into real systems, as well as with their department's efforts to increase the number and diversity of people doing research around cybersecurity.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.