EAGER: Towards Adversarial Attack Resistant Machine Learning Systems

基于机器学习的模式分类和深度学习等相关进展已经在从计算机视觉到医疗诊断的多个应用领域展示了令人印象深刻的能力。然而，也有研究表明，创建可以欺骗机器学习模型的对抗性输入是相对简单的。该项目的目标是为机器学习模型开发防御措施，即使面对复杂和坚定的对手也是强大的。该项目将对机器学习系统的安全性产生广泛的影响，推进跨学科研究，并促进本科生和代表性不足的群体参与计算机工程研究和教育。该项目将追求两条旨在阻止对抗输入生成中使用的梯度上升技术的防线。第一道防线将在输出置信水平中添加受控噪声，以拒绝对手访问精确的分类边界，同时寻求保持模型的准确性。第二道防线将在查询步骤中从多个训练模型中选择一个随机模型，这些模型具有相似的分类准确性，但内部参数和置信度不同。为了测试防御的有效性，该项目还将开发一个灰箱模型，以加速从黑盒机器学习模型生成对抗性输入。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning based pattern classification and related advances like deep learning have demonstrated impressive capabilities in multiple application domains, ranging from computer vision to medical diagnosis. However, it has also been shown that it is relatively straight-forward to create adversarial inputs that can fool machine learning models. The goal of this project is to develop defenses for machine learning models that are robust even in the face of sophisticated and determined adversaries. This project will have broad impact on the security of machine learning systems, advance cross-disciplinary research, and promote participation of undergraduates and under-represented groups in computer engineering research and education.This project will pursue two lines of defenses designed to hinder gradient ascent techniques used in adversarial input generation. The first line of defense will add controlled noise to output confidence levels to deny an adversary access to the precise classification boundary, while seeking to preserve model accuracy. The second line of defense will pursue choosing a random model in a query step from a pool of multiple trained models which have similar classification accuracy but differ in internal parameters and confidence levels. To test effectiveness of defenses, this project will also develop a gray-box model for accelerating adversarial input generation from a black-box machine learning model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.