IVF-Net: An Infrared and Visible Data Fusion Deep Network for Traffic Object Enhancement in Intelligent Transportation Systems
IVF-Net: An Infrared and Visible Data Fusion Deep Network for Traffic Object Enhancement in Intelligent Transportation Systems
复制标题
IVF-Net：用于智能交通系统中交通对象增强的红外和可见光数据融合深度网络
DOI：
10.1109/tits.2022.3210693
复制
发表时间：
2023-01
影响因子：
8.5
通讯作者：
Dengyin Zhang
中科院分区：
文献类型：
--
作者：
Mingye Ju;Chunming He;Juping Liu;Bin Kang;JIAN SU;Dengyin Zhang
红外与可见光数据融合(IVF)的目的是产生一种融合输出，同时突出突出的热辐射特征和保留纹理信息，不仅能够掌握交通运动所需的信息，而且能够突出智能交通系统(ITS)中需要躲避的不可见物体。因此，体外受精能够提高对各种挑战性交通情况的环境感知能力，例如雾天场景、多雨环境和弱光照明。然而，现有的体外受精算法不能提供一种将先验知识和网络结构集成到一个统一模型中的理论方式。此外，它们总是无法处理不同分辨率的红外和可见光数据对，这在现实的智能交通系统场景中是常见的。为此，本研究开发了一种新的模型启发的无监督网络，称为IVF-Net。具体地说，首先建立了增强的体外受精模型(IVFM)，该模型更加关注细节纹理信息和显著对象。根据近邻梯度理论，将该模型映射到具有可学习特征提取参数的深度网络，旨在吸取融合模型和深度学习的优点，更好地描述试管受精任务。最后，设计了一个多任务驱动的损失函数来训练映射网络。与以往的工作不同，我们的IVF-Net是由IVFM驱动的，其中每一层都具有语义可解释性和明确的任务，从而导致显著增强的融合效果。另一个优点是它只由简单的卷积结构组成，这保证了它的轻量化和效率。实验表明，IVF-Net具有更强的捕获关键交通信息的能力，突出了隐身对象的显著特征，是提高ITS后续应用可靠性的极佳候选。
Infrared and visible data fusion (IVF) aims to generate a fused output that simultaneously highlights salient thermal radiation features and preserves texture information, which can not only grasp the necessary information for traffic movement, but also highlight the invisible objects that need to be dodged in intelligent transportation system (ITS). Therefore, IVF is capable of improving the environmental perception ability for various challenging traffic situations, e.g., foggy scenarios, rainy environments, and low-light illumination. However, current available IVF algorithms cannot offer a theoretical manner to integrate a priori knowledge and the network structure into a unified model. Moreover, they always fail to handle infrared and visible data pairs with different resolutions, which is a common occurrence in real ITS scenarios. To this end, this study develops a novel model-inspired unsupervised network termed IVF-Net. Specifically, an enhanced IVF model (IVFM), which pays more attention on detailed texture information and salient objects, is first established. According to proximal gradient theory, then we map this model into a deep network with learnable feature extraction parameters, aiming to draw on the strengths of the fusion model and deep learning to better describe the IVF task. Finally, a multiple task-driven loss function is designed to train the mapped network. Unlike previous work, our IVF-Net is motivated by IVFM, each layer in which has a semantic interpretability and a clear mission, thereby leading to a significantly enhanced fusion effect. Another advantage is that it is only composed of simple convolution-based structures, which ensures its lightweight and efficiency. Experiments demonstrate that IVF-Net can have a stronger ability to capture the key traffic information and highlight the salient feature of imperceptible objects, which makes it an excellent candidate to improve the reliability of subsequent applications in ITS.
登录
查看更多内容
一种基于图像分解和稀疏表示的多模态图像融合新方法
DOI：
10.1016/j.ins.2017.09.010
发表时间：
2017-09
期刊：
Information Science
影响因子：
--
作者：
Zhiqing Zhu;Hongpeng Yin;Yi Chai;Guanqiu Qi
通讯作者：
Guanqiu Qi
DOI：
--
发表时间：
2016-03
期刊：
ArXiv
影响因子：
--
作者：
Vincent Dumoulin;Francesco Visin
通讯作者：
Vincent Dumoulin;Francesco Visin
影响因子：
5
作者：
Masi, Giuseppe;Cozzolino, Davide;Scarpa, Giuseppe
通讯作者：
Scarpa, Giuseppe
DOI：
10.1109/lpe.2001.945371
发表时间：
2001-08
期刊：
ISLPED'01: Proceedings of the 2001 International Symposium on Low Power Electronics and Design (IEEE Cat. No.01TH8581)
影响因子：
--
作者：
D. Markovic;B. Nikolić;R. Brodersen
通讯作者：
D. Markovic;B. Nikolić;R. Brodersen
DOI：
--
发表时间：
2019-05
期刊：
ArXiv
影响因子：
--
作者：
Sulaiman A. Alghunaim;K. Yuan;A. H. Sayed
通讯作者：
Sulaiman A. Alghunaim;K. Yuan;A. H. Sayed