Pedestrian Detection for Autonomous Cars: Inference Fusion of Deep Neural Networks
Pedestrian Detection for Autonomous Cars: Inference Fusion of Deep Neural Networks
复制标题
DOI：
10.1109/tits.2022.3210186
复制
发表时间：
2022-12
影响因子：
8.5
通讯作者：
M. Islam;vvAbdullah Al Redwan Newaz;A. Karimoddini
中科院分区：
文献类型：
--
作者：
M. Islam;vvAbdullah Al Redwan Newaz;A. Karimoddini
网络融合最近已被探索作为一种方法，用于提高行人检测性能。然而，大多数现有的融合方法遭受运行时的效率，模块化，可扩展性和可维护性，由于整个融合模型的复杂结构，其端到端的训练要求，和顺序的融合过程。针对这些挑战，本文提出了一种新的融合框架，结合对象检测器和语义分割网络的非对称推理，共同检测多个行人。这是通过引入一种基于共识的评分方法来实现的，该方法融合了来自对象检测器和语义分割网络的成对像素相关信息，以提高最终的置信度分数。在所提出的框架中的对象检测和语义分割网络的并行实现需要低的运行时开销。通过在公共数据集上融合不同的最先进的行人检测器和语义分割网络，广泛评估了所提出的融合框架的效率和鲁棒性。融合模型的推广也检查通过自动驾驶汽车收集的新的交叉行人数据。实验结果表明，该融合方法在实现有竞争力的运行效率的同时，显著提高了检测性能.
Network fusion has been recently explored as an approach for improving pedestrian detection performance. However, most existing fusion methods suffer from runtime efficiency, modularity, scalability, and maintainability due to the complex structure of the entire fused models, their end-to-end training requirements, and sequential fusion process. Addressing these challenges, this paper proposes a novel fusion framework that combines asymmetric inferences from object detectors and semantic segmentation networks for jointly detecting multiple pedestrians. This is achieved by introducing a consensus-based scoring method that fuses pair-wise pixel-relevant information from the object detector and the semantic segmentation network to boost the final confidence scores. The parallel implementation of the object detection and semantic segmentation networks in the proposed framework entails a low runtime overhead. The efficiency and robustness of the proposed fusion framework are extensively evaluated by fusing different state-of-the-art pedestrian detectors and semantic segmentation networks on a public dataset. The generalization of fused models is also examined on new cross pedestrian data collected through an autonomous car. Results show that the proposed fusion method significantly improves detection performance while achieving competitive runtime efficiency.