Robustness of Deep Learning Perception Models

EPSRC领域：人工智能技术，人工智能和机器人主题，验证和正确性对研究背景的简要描述，包括潜在影响：深度学习（DL）近年来取得了巨大的进步，导致该技术在现实世界的应用得到了广泛的应用，包括人脸识别和自动驾驶，但感知模型在对抗性示例中可能不稳定，其中对输入的微小修改会导致错误分类。为了确保应用程序的安全性，需要严格的方法来促进健壮的感知模型的开发，这些模型可以合并到机器人应用的自动控制器中。评估深度学习模型鲁棒性的典型方法是通过启发式搜索对抗性示例（例如，基于梯度的随机搜索），如果没有找到对抗性示例，则不能保证不存在。另一种更强大的方法是采用自动验证，其目的是为给定场景中的模型行为提供可证明的保证。虽然最近在这个方向上取得了很大进展，但重点主要集中在简单输入操作的局部鲁棒性上，并且缺乏支持自动驾驶应用中典型的自然几何变换和上下文效果的框架。深度学习感知模型的成功部署关键取决于能够保证其对广泛的自然和上下文转换的鲁棒性。该项目将开发新的方法，为深度学习感知模型提供这种语义鲁棒性保证。目的和目标：在本研究计划中，我们将以PI开发的深度学习技术（例如深度神经网络的安全验证，CAV 2017）及其对贝叶斯神经网络的扩展（例如贝叶斯神经网络的概率安全性，UAI 2020）为基础，开发用于评估可包含在自动车辆和机器人控制器中的3D感知模型的鲁棒性的理论基础。该研究将以具体的应用场景和传感框架为基础，并将利用一系列符号和神经技术，包括贝叶斯学习、不确定性量化、约束求解和因果推理。重点将放在调查研究问题和开发与实际应用相关的框架上。研究方法的新颖性：本项目将重点关注表征学习和语义鲁棒性之间的交集。大多数先前的工作集中在有界像素扰动下的分类鲁棒性。我们假设语义鲁棒性是一个更强的鲁棒性条件，它不仅要求扰动下稳定的标签预测，而且要求输入的中间表示在语义扰动下是稳定的。因此，我们将重点关注表征学习的鲁棒性，特别是在复杂的几何变换和上下文变化方面。

EPSRC areas: Artificial intelligence technologies, Artificial intelligence and robotics theme, Verification and correctnessCompany involved: Toyota EuropeBrief description of the context of the research including potential impact:Deep learning (DL) has advanced dramatically in recent years, resulting in widespread take up of the technology in real-world applications, including face recognition and autonomous driving, but perception models can be unstable wrt adversarial examples, where a small modification to input causes a misclassification. To ensure safety and security of applications, rigorous methodologies are needed that facilitate the development of robust perception models that can be incorporated within automated controllers for robotic applications. A typical approach to evaluating robustness of DL models is through heuristic search for adversarial examples (e.g., gradient-based, stochastic search), which offers no guarantees that adversarial examples do not exist if not found. An alternative, more powerful, method is to employ automated verification, which aims to provide provable guarantees on the model behaviour in a given scenario. While there has been much progress recently in this direction, the focus has been mainly on local robustness with respect to simple input manipulations, and there is a lack of frameworks that support natural geometric transformations and contextual effects that are typical in autonomous driving applications. Successful deployment of DL perception models crucially depends on the ability to provably guarantee their robustness against a wide range of natural and contextual transformations. This project will develop new methods for providing such semantic robustness guarantees for DL perception models.Aims and Objectives:In this research programme we will build on the techniques developed by the PI for deep learning (e.g. Safety Verification of Deep Neural Networks, CAV 2017) and their extension to Bayesian neural networks (e.g. Probabilistic Safety for Bayesian Neural Networks, UAI 2020) to develop the theoretical foundations for evaluating robustness of 3D perception models that are amenable to inclusion within automated vehicle and robotic controllers. The research will be informed by concrete application scenarios and sensing frameworks, and will draw on a range of symbolic and neural techniques, including Bayesian learning, uncertainty quantification, constraint solving and causal reasoning. The focus will be on investigating research questions and developing frameworks that are relevant for practical applications. Novelty of the research methodology:This project will focus on the intersection between representation learning and semantic robustness. Most prior work has focused on classification robustness under bounded pixel perturbations. We posit that semantic robustness is a stronger robustness condition that requires not only stable label predictions under perturbations but also that the intermediate representations of an input are stable under semantic perturbations. Therefore, we will focus on the robustness of representation learning, particularly as regarding complex geometric transformations and contextual changes.