3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing
3D Pedestrian Detection in Farmland by Monocular RGB Image and Far-Infrared Sensing
复制标题
DOI：
10.3390/rs13152896
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Wei Tian;Zhenwen Deng;Dong Yin;Zehan Zheng;Yuyao Huang;Xin Bi
中科院分区：
文献类型：
--
作者：
Wei Tian;Zhenwen Deng;Dong Yin;Zehan Zheng;Yuyao Huang;Xin Bi
农业机械的自动驾驶对提高农业生产效率具有重要意义，但由于昼夜环境条件变化很大，因此仍然具有挑战性。为解决农田中行人的操作安全问题，提出了一种基于单目RGB和远红外（FIR）图像的三维人体感知方法。由于农业三维行人检测的公共数据集是稀缺的，提出了一个新的数据集，命名为“FieldSafePedestrian”，其中包括在白天和夜间的字段图像。此外，还详细阐述了夜间图像的数据增强和半自动标注方法，以方便行人的3D标注。针对光轴不平行传感器的异构图像融合问题，提出了双输入深度引导的动态依赖扩张融合网络（D5 F），该网络利用深度信息辅助FIR图像和RGB图像的像素对齐，并采用动态滤波来引导异构信息融合。在白天和夜间的现场图像上的实验表明，与现有技术相比，动态对准图像融合在中心距离和BEV-IOU方面分别实现了3.9%和4.5%的精度增益，而不影响运行时效率。
The automated driving of agricultural machinery is of great significance for the agricultural production efficiency, yet is still challenging due to the significantly varied environmental conditions through day and night. To address operation safety for pedestrians in farmland, this paper proposes a 3D person sensing approach based on monocular RGB and Far-Infrared (FIR) images. Since public available datasets for agricultural 3D pedestrian detection are scarce, a new dataset is proposed, named as “FieldSafePedestrian”, which includes field images in both day and night. The implemented data augmentations of night images and semi-automatic labeling approach are also elaborated to facilitate the 3D annotation of pedestrians. To fuse heterogeneous images of sensors with non-parallel optical axis, the Dual-Input Depth-Guided Dynamic-Depthwise-Dilated Fusion network (D5F) is proposed, which assists the pixel alignment between FIR and RGB images with estimated depth information and deploys a dynamic filtering to guide the heterogeneous information fusion. Experiments on field images in both daytime and nighttime demonstrate that compared with the state-of-the-arts, the dynamic aligned image fusion achieves an accuracy gain of 3.9% and 4.5% in terms of center distance and BEV-IOU, respectively, without affecting the run-time efficiency.