A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data
A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data
复制标题
DOI：
10.1109/smc52423.2021.9658639
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
M. Islam;Abdullah Al Redwan Newaz;A. Karimoddini
中科院分区：
文献类型：
--
作者：
M. Islam;Abdullah Al Redwan Newaz;A. Karimoddini
提出了一种融合摄像机和激光雷达传感器数据的行人检测与跟踪方法。为了应对自动驾驶场景带来的挑战，提出了一种集成的跟踪和检测框架。检测阶段通过将LiDAR流转换为计算上容易处理的深度图像来执行，然后，开发深度神经网络来识别RGB图像和深度图像中的行人候选。为了提供准确的信息，通过使用卡尔曼滤波融合多模式传感器信息来进一步增强检测阶段。跟踪阶段是卡尔曼滤波预测和光流算法的结合，用于跟踪场景中的多个行人。我们在真实的公共驾驶数据集上对我们的框架进行了评估。实验结果表明，与单纯使用基于图像的行人检测方法相比，该方法的性能有了显著的提高。
This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. To deal with the challenges associated with the autonomous driving scenarios, an integrated tracking and detection framework is proposed. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates both in RGB and depth images. To provide accurate information, the detection phase is further enhanced by fusing multi-modal sensor information using the Kalman filter. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene. We evaluate our framework on a real public driving dataset. Experimental results demonstrate that the proposed method achieves significant performance improvement over a baseline method that solely uses image-based pedestrian detection.