CRII: RI: Immune-Inspired Learning Foundations of Neural Network General Robustness

在神经网络（NN）快速发展的推动下，人工智能在许多领域取得了显着的成功。然而，人类不可见的小扰动可以故意添加到输入中，导致NN做出错误的预测。更重要的是，攻击者甚至可以自定义不同的扰动策略来绕过现有NN的学习方法和防御。因此，一个悬而未决的问题是如何使NN对多种类型的这种对抗性扰动更加鲁棒。人类有一个高度进化的免疫系统，可以抵御多种威胁，即使是以前从未遇到过的威胁。受强大的免疫系统的启发，该项目旨在将关键的免疫系统原理注入NN中，以提高其一般鲁棒性，即抵御多种类型扰动的能力。这些研究成果将使需要强大NN的领域受益，例如公共卫生和自动驾驶。此外，该项目计划支持跨学科的教育和研究项目（涉及机器学习和生物学），并向高中，特别是来自代表性不足群体的学生开展推广活动。为了缩小现有以机器为中心的鲁棒学习框架和鲁棒免疫模型之间的巨大差距，该项目的重点是在神经网络设计中加入三个强大的免疫系统组件，以帮助神经网络抵御各种攻击，并不断强化自己。拟议的研究包括三个目标。第一个目标是开发一种免疫启发的人口点混合优化，可以有效地搜索鲁棒的解决方案，并通过自对抗模式连接策略保持搜索效率。开发的技术将改善现有的基于点的学习方法，很容易陷入坏的局部极小值。第二个目标从鲁棒免疫共识的角度考虑神经网络学习，将随机性纳入学习，允许NN捕获全局特征信息。第三个目标扩展了前两个目标，允许NN通过免疫系统启发的终身学习机制来适应不可预见类型的对抗性攻击，该机制包括热启动防御策略和基于知识蒸馏的记忆模型更新。这项研究工作将使人们对强大的机器学习有一个新的理解和设计框架。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Driven by rapid advances in neural networks (NNs), artificial intelligence has achieved remarkable success in many fields. However, small perturbations invisible to humans can be purposely added to inputs to cause NNs to make incorrect predictions. What is more, attackers can even customize different perturbation strategies to bypass existing NNs' learning methods and defenses. Thus, one open question is how to make NNs more robust to multiple types of such adversarial perturbations. Humans have a highly evolved immune system that can defend against multiple threats, even those never encountered before. Inspired by the powerful immune system, this project aims to infuse key immune system principles into NNs to improve their general robustness, that is, their capability to defend against multiple types of perturbations. The research outcomes will benefit fields that demand robust NNs, such as public health and autonomous driving. Furthermore, this project is planned to support cross-disciplinary education and research projects (involving machine learning and biology) for both undergraduate and graduate students, with outreach activities to high schools and particularly students from underrepresented groups.To reduce the substantial gap between existing machine-centric robust learning frameworks and robust immune models, this project focuses on incorporating into neural network design three robust immune-system components to help neural networks defend themselves against various attacks and continuously harden themselves. The proposed research consists of three aims. The first aim is to develop an immune-inspired population-point hybrid optimization that can effectively search for robust solutions and maintain the searching efficiency via a self-adversarial mode connectivity strategy. The developed technique will improve existing point-based learning approaches, which easily become trapped in bad local minima. The second aim considers neural network learning from a robust immune consensus perspective that incorporates stochasticity into learning, allowing NNs to capture global feature information. The third aim expands the first two aims by allowing NNs to adapt to unforeseen types of adversarial attacks with an immune-system-inspired lifelong learning regime consisting of a warm start defense strategy and knowledge distillation-based memory model update. This research effort will enable a new understanding of and a design framework for robust machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.