Multimodal object detection using unsupervised transfer learning and adaptation techniques
Multimodal object detection using unsupervised transfer learning and adaptation techniques
复制标题
DOI：
10.1117/12.2532794
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Rachael Abbott;N. Robertson;Jesús Martínez Del Rincón;B. Connor
中科院分区：
文献类型：
--
作者：
Rachael Abbott;N. Robertson;Jesús Martínez Del Rincón;B. Connor
深度神经网络在使用RGB数据的对象检测任务上实现了最先进的性能。然而，在国防和安全行动中，使用多模态图像进行检测有许多优点。例如，红外模式提供持续的监控，在光线不好的条件下和24小时运行时至关重要。因此，创建一个可以使用红外图像的目标检测系统至关重要。收集和标记大量的热成像是非常昂贵和耗时的。因此，我们建议动员标记的RGB数据，以实现在IR模态检测。在本文中，我们提出了一种使用无监督迁移学习和自适应技术的多模态对象检测方法。我们在RGB图像上训练更快的RCNN，并使用热像仪进行测试。这些图像包含对象类;人和陆地车辆，并代表包括混乱和遮挡的真实场景。通过使用额外的损失函数进行训练，我们将基线F1分数提高了20%，这减少了RGB和IR特征图之间的差异。这项工作表明，无监督的模态自适应是可能的，我们有机会最大限度地利用标记的RGB图像在多种模态中进行检测。这项工作的新奇之处包括：使用红外图像，从RGB到红外的模态适应，用于物体检测，以及在不受控制的环境中使用真实图像的能力。这项工作对国防和安全界的实际影响是提高了性能，节省了数据收集和注释的时间和金钱。
Deep neural networks achieve state-of-the-art performance on object detection tasks with RGB data. However, there are many advantages of detection using multi-modal imagery for defence and security operations. For example, the IR modality offers persistent surveillance and is essential in poor lighting conditions and 24hr operation. It is, therefore, crucial to create an object detection system which can use IR imagery. Collecting and labelling large volumes of thermal imagery is incredibly expensive and time-consuming. Consequently, we propose to mobilise labelled RGB data to achieve detection in the IR modality. In this paper, we present a method for multi-modal object detection using unsupervised transfer learning and adaptation techniques. We train faster RCNN on RGB imagery and test with a thermal imager. The images contain object classes; people and land vehicles and represent real-life scenes which include clutter and occlusions. We improve the baseline F1-score by up to 20% through training with an additional loss function, which reduces the difference between RGB and IR feature maps. This work shows that unsupervised modality adaptation is possible, and we have the opportunity to maximise the use of labelled RGB imagery for detection in multiple modalities. The novelty of this work includes; the use of the IR imagery, modality adaption from RGB to IR for object detection and the ability to use real-life imagery in uncontrolled environments. The practical impact of this work to the defence and security community is an increase in performance and the saving of time and money in data collection and annotation.