Robust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals
Robust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals
复制标题
DOI：
10.1109/cvpr46437.2021.00051
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Kun Qian;Shilin Zhu;Xinyu Zhang;Erran L. Li
中科院分区：
文献类型：
--
作者：
Kun Qian;Shilin Zhu;Xinyu Zhang;Erran L. Li
利用激光雷达和摄像头等视觉传感器进行车辆检测是实现自动驾驶的关键功能之一。虽然它们在良好的天气条件下生成具有丰富信息的细粒度点云或高分辨率图像，但在不透明粒子会扭曲光线并显著降低能见度的恶劣天气(如雾)中，它们会失败。因此，依赖激光雷达或相机的现有方法在罕见但严重的不利天气条件下会经历显著的性能下降。为了解决这一问题，我们采用了辅助雷达，这种雷达受恶劣天气的影响较小，在车辆上变得普遍。本文提出了一种多模式车辆检测网络(MVDNet)，它是一种两级深度融合检测器，它首先从两个传感器生成建议，然后融合多模式传感器流之间的区域特征来改善最终的检测结果。为了评估MVDNet，我们基于从开源牛津雷达RobotCar收集的原始激光雷达和雷达信号创建了一个程序化生成的训练数据集。结果表明，MVDNet在平均精度(AP)方面明显优于其他最先进的方法，特别是在恶劣天气条件下。代码和数据可在https://github.com/qiank10/MVDNet.上获得
Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving. While they generate fine-grained point clouds or high-resolution images with rich information in good weather conditions, they fail in adverse weather (e.g., fog) where opaque particles distort lights and significantly reduce visibility. Thus, existing methods relying on lidar or camera experience significant performance degradation in rare but critical adverse weather conditions. To remedy this, we resort to exploiting complementary radar, which is less impacted by adverse weather and becomes prevalent on vehicles. In this paper, we present Multimodal Vehicle Detection Network (MVDNet), a two-stage deep fusion detector, which first generates proposals from two sensors and then fuses region-wise features between multimodal sensor streams to improve final detection results. To evaluate MVDNet, we create a procedurally generated training dataset based on the collected raw lidar and radar signals from the open-source Oxford Radar Robotcar. We show that the proposed MVDNet surpasses other state-of-the-art methods, notably in terms of Average Precision (AP), especially in adverse weather conditions. The code and data are available at https://github.com/qiank10/MVDNet.