Learning Distilled Collaboration Graph for Multi-Agent Perception
Learning Distilled Collaboration Graph for Multi-Agent Perception
复制标题
DOI：
--
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Yiming Li;Shunli Ren;Pengxiang Wu;Siheng Chen;Chen Feng;Wenjun Zhang
中科院分区：
文献类型：
--
作者：
Yiming Li;Shunli Ren;Pengxiang Wu;Siheng Chen;Chen Feng;Wenjun Zhang
为了在多智能体感知中实现更好的性能 - 带宽权衡，我们提出了一种新颖的蒸馏协作图（DiscoGraph）来对智能体之间可训练、具有姿态感知和自适应的协作进行建模。我们的关键创新在于两个方面。首先，我们提出了一个师生框架，通过知识蒸馏来训练DiscoGraph。教师模型采用具有整体视图输入的早期协作；学生模型基于具有单视图输入的中间协作。我们的框架通过约束学生模型中的协作后特征图以匹配教师模型中的对应关系来训练DiscoGraph。其次，我们在DiscoGraph中提出了一种矩阵值的边权重。在这样一个矩阵中，每个元素都反映了特定空间区域内智能体间的注意力，使智能体能够自适应地突出信息丰富的区域。在推理过程中，我们只需要使用被称为蒸馏协作网络（DiscoNet）的学生模型。由于师生框架，共享DiscoNet的多个智能体能够协作达到具有整体视图的假设教师模型的性能。我们的方法在V2X - Sim 1.0上得到了验证，V2X - Sim 1.0是我们使用CARLA和SUMO联合模拟合成的一个大规模多智能体感知数据集。我们在多智能体3D目标检测中的定量和定性实验表明，DiscoNet不仅能够比最先进的协作感知方法实现更好的性能 - 带宽权衡，而且还带来了更直接的设计原理。我们的代码可在https://github.com/ai4ce/DiscoNet获取。
To promote better performance-bandwidth trade-off for multi-agent perception, we propose a novel distilled collaboration graph (DiscoGraph) to model trainable, pose-aware, and adaptive collaboration among agents. Our key novelties lie in two aspects. First, we propose a teacher-student framework to train DiscoGraph via knowledge distillation. The teacher model employs an early collaboration with holistic-view inputs; the student model is based on intermediate collaboration with single-view inputs. Our framework trains DiscoGraph by constraining post-collaboration feature maps in the student model to match the correspondences in the teacher model. Second, we propose a matrix-valued edge weight in DiscoGraph. In such a matrix, each element reflects the inter-agent attention at a specific spatial region, allowing an agent to adaptively highlight the informative regions. During inference, we only need to use the student model named as the distilled collaboration network (DiscoNet). Attributed to the teacher-student framework, multiple agents with the shared DiscoNet could collaboratively approach the performance of a hypothetical teacher model with a holistic view. Our approach is validated on V2X-Sim 1.0, a large-scale multi-agent perception dataset that we synthesized using CARLA and SUMO co-simulation. Our quantitative and qualitative experiments in multi-agent 3D object detection show that DiscoNet could not only achieve a better performance-bandwidth trade-off than the state-of-the-art collaborative perception methods, but also bring more straightforward design rationale. Our code is available on https://github.com/ai4ce/DiscoNet.