VIPAuto: Robust and Adaptive Visual Perception for Automated Vehicles in Complex Dynamic Scenes

自动驾驶汽车（AVs）在将现有交通系统转变为智能生态系统方面具有巨大潜力，可以增强道路安全、服务可及性和环境可持续性。然而，这种潜力受到当前基于学习的视觉感知（VP）系统的无能的阻碍，该系统是从有限的标记数据中训练的，因此无法理解复杂的动态驾驶场景。为了解决这个问题，VIPAuto旨在开发一系列突破性的技术，以创建一个通用的VP系统，弥合有限的训练数据与现实场景中无穷无尽的变化之间的差距。为此，将解决两个重大挑战：1)提高VP系统在恶劣天气条件下的场景理解精度；2)使VP系统能够识别并逐步学习异常物体。为了解决第一个挑战，将开发一种自监督领域适应策略，使VP模型能够通过将知识从晴朗天气领域转移到恶劣天气领域来从未标记的数据中学习，这是通过创新地建立域间和域内共同知识来实现的。为了解决第二个挑战，我们将创建一个少量增量学习策略，通过设计对比学习来排斥已知类中的未知对象，并创建一个基于高级认知理论的表示来提高从少数样本中学习的能力，从而使VP模型能够学习未知对象。所提出的解决方案将被集成到一个优化的VP系统中，并在复杂的动态驾驶场景下进行评估。VIPAuto将为渐进式自适应副总裁技术提供理论基础和实践技术，从而提高现实世界中场景理解的鲁棒性，以支持自动驾驶汽车的决策，并为欧盟到2050年实现“零愿景”（零道路死亡事故）的长期目标做出贡献。

Automated Vehicles (AVs) have great potential in revolutionising the existing transportation system into an intelligent ecosystem that can enhance road safety, service accessibility and environmental sustainability. However, this potential is hampered by the inability of the current learning-based visual perception (VP) system that is trained from limited labelled data and thus fails to understand the complex dynamic driving scene. To deal with this problem, VIPAuto aims to develop a series of ground-breaking technologies for creating a generalized VP system and bridging the gap between the limited training data and the endless variations in the real scene. To this end, two significant challenges will be addressed: 1) boosting the scene understanding accuracy of the VP system under adverse weather conditions and 2) enabling the VP system to recognize and incrementally learn anomalous objects. To tackle the first challenge, a self-supervised domain adaptation strategy will be developed to enable the VP model to learn from unlabelled data by transferring knowledge from the clear weather domain to the adverse weather domain, which is empowered by innovatively established inter- and intra-domain common knowledge. To tackle the second challenge, a few-shot incremental learning strategy will be created to enable the VP model to learn unknown objects by designing contrastive learning to repel unknown objects from known classes and creating an advanced cognitive theory-based representation to promote learning capacity from a few samples. The proposed solutions will be integrated into an optimized VP system and evaluated under the complex dynamic driving scene. VIPAuto will provide theoretical foundations and practical techniques for incrementally adaptive VP technologies, thereby promoting the robustness of scene understanding in the real world to support the decision-making of AVs, and contributing to the EU's long-term goal of "Vision Zero" (zero road fatalities) by 2050.