Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network
Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network
复制标题
DOI：
10.3390/rs10050800
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Jingxiang Yang;Yongqiang Zhao;J. Chan
中科院分区：
文献类型：
--
作者：
Jingxiang Yang;Yongqiang Zhao;J. Chan
提高高光谱图像的空间分辨率具有重要的应用价值。高分辨率多光谱图像与高光谱图像的融合是高光谱图像增强的一项重要技术。受深度学习在图像增强中的成功启发，本文通过设计具有两个分支的深度卷积神经网络（CNN），提出了一种HSI-MSI融合方法，该方法专门用于HSI和MSI的特征。为了利用光谱相关性并融合MSI，我们使用两个CNN分支从低分辨率HSI中每个像素的光谱及其对应的MSI空间邻域中提取特征。然后将提取的特征连接并馈送到全连接（FC）层，在那里HSI和MSI的信息可以完全融合。FC层的输出是预期HR HSI的频谱。在实验中，我们评估所提出的方法对机载可见光红外成像光谱仪（AVIRIS），和环境测绘和分析程序（EnMAP）数据。并将其应用于真实的Hyperion-Sentinel数据融合。仿真数据和真实的数据的融合结果表明，该方法与其他先进的融合方法相比具有竞争力。
Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.