Combining global receptive field and spatial spectral information for single-image hyperspectral super-resolution
Combining global receptive field and spatial spectral information for single-image hyperspectral super-resolution
复制标题
DOI：
10.1016/j.neucom.2023.126277
复制
发表时间：
2023-04
期刊：
影响因子：
6
通讯作者：
Yiming Wu;Ronghui Cao;Yikun Hu;Jin Wang;KenLi Li
中科院分区：
文献类型：
--
作者：
Yiming Wu;Ronghui Cao;Yikun Hu;Jin Wang;KenLi Li
由于缺乏辅助图像，单图像超光谱在空间维度上的重建性能不如融合图像超光谱。一些研究尝试使用3D卷积来探索空间光谱之间的隐藏特征以增强空间细节。然而，无论是二维还是三维卷积，得到的感受野都是有限的，忽略了全局空间信息对高光谱图像重建的影响，不能用于长程相关建模。为此，本文首次尝试将联合收割机Transformer与三维卷积相结合应用于单幅图像的高光谱超分辨，提出了一种基于三维卷积和Transformer的高光谱超分辨（3D-THSR）网络，该网络在获取空间整体感受野的同时，探索了空间与光谱之间隐藏的信息。具体而言，Transformer模块用于特征提取，以增强全局空间信息和长距离特征的学习能力。此外，三维卷积模块嵌入在Transformer模块中，通过融合光谱和空间维度来提取不同光谱波段之间的信息。最后，我们设计了三种损失函数来训练网络，以减少失真谱，保证谱带纯度。通过6个高光谱评价指标、空间细节图像、光谱误差线图和烧蚀研究，与其他单图像高光谱方法进行比较，证明该方法具有更好的高光谱超分辨率重建性能。
Single-image hyperspectral super-resolution has poorer reconstruction performance in the spatial dimension than fused-image hyperspectral super-resolution due to the lack of auxiliary images. Some studies have attempted to use 3D convolution to explore hidden features between spatial spectra to enhance spatial details. However, either 2D or 3D convolution, the obtained receptive fields are limited, ignoring the effect of global spatial information on hyperspectral image reconstruction, and cannot be used for long-range dependent modeling. Therefore, we make the first attempt to combine Transformer with 3D convolution in single-image hyperspectral super-resolution and propose a 3D convolution and Transformer hyperspectral super-resolution (3D-THSR) network, which explores the hidden information between space and spectra while obtaining the global receptive field of space. Specifically, the Transformer module is used for feature extraction to enhance the learning ability of global spatial information and long-distance features. In addition, the 3D convolution module is embedded in the Transformer module to extract the information between different spectral bands by fusing the spectral and spatial dimensions. Finally, we design to train the network by using three loss functions to reduce the distortion spectrum and ensure the spectral band purity. Compared with other single-image hyperspectral methods by six hyperspectral evaluation metrics, spatial detail image, spectral error line map, and ablation study, it is proved that the proposed method achieves better hyperspectral super-resolution reconstruction performance.