Exploring mmWave Radar and Camera Fusion for High-Resolution and Long-Range Depth Imaging
Exploring mmWave Radar and Camera Fusion for High-Resolution and Long-Range Depth Imaging
复制标题
DOI：
10.1109/iros47612.2022.9982080
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Diana Zhang;Akarsh Prabhakara
中科院分区：
文献类型：
--
作者：
Diana Zhang;Akarsh Prabhakara
机器人地理围栏和监控系统需要对违反周界限制的物体进行准确监控。在本文中，我们寻求一种解决方案，可以从单个有利位置（例如安装在杆上的平台）在扩展范围（长达 300 米）内以高精度（几十厘米）对此类感兴趣的物体进行深度成像。不幸的是，关于单独使用相机、激光雷达和雷达的深度成像的丰富文献很难满足现实条件下的这些严格要求。本文提出了 Metamoran，这是一种通过融合毫米波雷达和摄像头这两种互补技术的优势来探索感兴趣物体的远程深度成像的解决方案。与相机不同，毫米波雷达即使在很远的距离内也能提供出色的厘米级深度分辨率。然而，它们的角分辨率至少比相机系统差 10 倍。融合这两种模式是很自然的，但在高杂波和长距离的场景中，雷达反射很弱，并且会出现虚假伪影。 Metamoran 的核心贡献是利用相机图像的图像分割和单目深度估计来帮助整理雷达并发现真实的物体反射。我们对 Metamoran 在 400 种不同场景下的深度成像能力进行了详细评估。我们的评估表明，Metamoran 可估计 90 m 以内的静态物体和 305 m 以内的移动物体的深度，中位误差为 28 cm，比简单的雷达+相机基线提高了 13 倍，比单目深度估计提高了 23 倍。
Robotic geo-fencing and surveillance systems require accurate monitoring of objects if/when they violate perimeter restrictions. In this paper, we seek a solution for depth imaging of such objects of interest at high accuracy (few tens of cm) over extended ranges (up to 300 meters) from a single vantage point, such as a pole mounted platform. Unfortunately, the rich literature in depth imaging using camera, lidar and radar in isolation struggles to meet these tight requirements in real-world conditions. This paper proposes Metamoran, a solution that explores long-range depth imaging of objects of interest by fusing the strengths of two complementary technologies: mmWave radar and camera. Unlike cameras, mmWave radars offer excellent cm-scale depth resolution even at very long ranges. However, their angular resolution is at least 10x worse than camera systems. Fusing these two modalities is natural, but in scenes with high clutter and at long ranges, radar reflections are weak and experience spurious artifacts. Metamoran's core contribution is to leverage image segmentation and monocular depth estimation on camera images to help declutter radar and discover true object reflections. We perform a detailed evaluation of Metamoran's depth imaging capabilities in 400 diverse scenarios. Our evaluation shows that Metamoran estimates the depth of static objects up to 90 m away and moving objects up to 305 m away and with a median error of 28 cm, an improvement of 13 x over a naive radar+camera baseline and 23 x compared to monocular depth estimation.