Sniffer Faster R-CNN ++: An Efficient Camera-LiDAR Object Detector with Proposal Refinement on Fused Candidates
Sniffer Faster R-CNN ++: An Efficient Camera-LiDAR Object Detector with Proposal Refinement on Fused Candidates
复制标题
DOI：
10.1145/3631138
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Sudip Dhakal;Dominic Carrillo;Deyuan Qu;Qing Yang;Song Fu
中科院分区：
文献类型：
--
作者：
Sudip Dhakal;Dominic Carrillo;Deyuan Qu;Qing Yang;Song Fu
在本文中，我们提出了Sniffer Faster r - cnn++，这是一种高效的摄像头-激光雷达后期融合网络，用于自动驾驶场景下的低复杂度和准确的目标检测。所提出的检测网络架构对任意三维（3D）探测器的输出候选点和任意二维探测器区域建议网络的建议进行操作，以生成最终的预测结果。与单模态目标检测方法相比，基于融合的方法在许多情况下存在不同数据集成的困难。一方面，基于融合的网络模型复杂，另一方面，它需要大量的计算开销和资源，特别是训练和推理的处理管道，早期融合和深度融合方法。因此，我们设计了一种后期融合网络，该网络与预训练的单模态检测器在不改变的情况下进行合作，仅在检测级别执行关联。除此之外，基于激光雷达的方法由于其稀疏性而无法检测到远距离目标，因此我们设计了建议改进算法来共同优化检测候选对象，辅助远距离目标的检测。在具有挑战性的KITTI数据集的3D和2D检测基准上进行的大量实验表明，我们提出的网络架构显著提高了检测精度，加快了检测速度。
In this article, we present Sniffer Faster R-CNN++, an efficient camera-LiDAR late fusion network for low complexity and accurate object detection in autonomous driving scenarios. The proposed detection network architecture operates on output candidates of any three-dimensional (3D) detector and proposals from regional proposal network of any 2D detector to generate final prediction results. In comparison to the single modality object detection approaches, fusion-based methods in many instances suffer from dissimilar data integration difficulties. On the one hand, fusion-based network models are complicated in nature and, on the other hand, they require large computational overhead and resources, processing pipelines for training and inference specially, the early fusion and deep fusion approaches. As such, we devise a late fusion network that in-cooperates pre-trained, single-modality detectors without change, performing association only at the detection level. In addition to this, lidar-based method fail to detect distant object due to its sparse nature so we devise proposal refinement algorithm to jointly optimize detection candidates and assist detection for distant objects. Extensive experiments on both the 3D and 2D detection benchmark of challenging KITTI dataset illustrate that our proposed network architecture significantly improves the detection accuracy, accelerating the detection speed.