SaTC: CORE: Small: Attack-Agnostic Defenses against Adversarial Inputs in Learning Systems

深度学习技术有望彻底改变人们的生活和工作方式。然而，深度学习系统天生就容易受到敌意输入的影响，这些输入是恶意制作的样本，用于触发深度神经网络的不当行为，从而在安全关键应用中导致灾难性后果。防御此类攻击的根本挑战来自于它们的自适应和可变性质：敌意输入是针对深层神经网络量身定做的，而精心设计的策略因具体攻击而有很大差异。这个项目开发了EagleEye，这是一个通用的攻击不可知的防御框架，它(I)有效地对抗不可见的攻击变体，(Ii)保留深度神经网络的预测能力，(Iii)补充现有的防御机制，以及(Iv)提供对深度学习输出中的潜在风险的全面诊断。特别是，EagleEye利用了大多数攻击背后的一组不变属性，包括“最小化原则”：为了最大限度地避免攻击，通过对合法输入应用最小可能的扭曲来生成对抗性输入。通过以有原则的方式利用这些特性，EagleEye有效地区分敌意输入(完整性检查)，甚至发现它们的正确输出(真相恢复)。具体的研究任务包括：(I)识别合法和敌意输入的内在不同属性(区分因素)，(Ii)基于这些区分因素开发攻击不可知的敌意输入检测方法，以及(Iii)分析敌手为逃避此类防御而可能采取的对策。这项研究不仅促进了深度学习驱动的系统和服务的采用，而且对总体上设计和实现健壮的机器学习系统也有启发作用。在这个项目中开发的新理论和系统被整合到本科和研究生教育中，并用于提高公众对机器学习安全重要性的认识。有关该项目的更多信息，请访问项目主页：http://x-machine.github.io/project/eagleeye

Deep learning technologies hold great promise to revolutionize the way people live and work. However, deep learning systems are inherently vulnerable to adversarial inputs, which are maliciously crafted samples to trigger deep neural networks to misbehave, leading to disastrous consequences in security-critical applications. The fundamental challenges of defending against such attacks stem from their adaptive and variable nature: adversarial inputs are tailored to target deep neural networks, while crafting strategies vary greatly with concrete attacks. This project develops EagleEye, a universal, attack-agnostic defense framework that (i) works effectively against unseen attack variants, (ii) preserves predictive power of deep neural networks, (iii) complements existing defense mechanisms, and (iv) provides comprehensive diagnosis about potential risks in deep learning outputs.In particular, EagleEye leverages a set of invariant properties underlying most attacks, including the "minimality principle": to maximize attack evasiveness, an adversarial input is generated by applying the minimum possible distortion to a legitimate input. By exploiting such properties in a principled manner, EagleEye effectively discriminates adversarial inputs (integrity checking) and even uncovers their correct outputs (truth recovery). The specific research tasks include: (i) identifying inherently distinct properties (differentiators) of legitimate and adversarial inputs, (ii) developing attack-agnostic adversarial input detection methods based on these differentiators, and (iii) analyzing possible countermeasures by adversaries to evade such defenses. This research not only facilitates the adoption of deep learning-powered systems and services, but also enlightens designing and implementing robust machine learning systems in general. New theories and systems developed in this project are integrated into undergraduate and graduate education and used to raise public awareness of the importance of machine learning security. More information about this project can be found at the project homepage: http://x-machine.github.io/project/eagleeye