A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-Resolution
A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-Resolution
复制标题
DOI：
10.1109/tgrs.2022.3193441
复制
发表时间：
2022-01-01
影响因子：
8.2
通讯作者：
Dancey, Darren
中科院分区：
文献类型：
--
作者：
Shi, Yue;Han, Liangxiu;Dancey, Darren
真实感高光谱图像（HSI）超分辨率（SR）技术旨在从低分辨率（LR）图像中生成具有更高光谱和空间保真度的高分辨率（HR）HSI。生成对抗网络（GAN）已被证明是图像SR的有效深度学习框架。然而，现有基于GAN的模型的优化过程经常遭受模式崩溃的问题，导致频谱空间不变重建的能力有限。这可能导致所生成的HSI的频谱-空间失真，特别是在大的放大因子的情况下。为了缓解模式崩溃的问题，这项工作提出了一种新的GAN模型耦合一个潜在的编码器（LE-GAN），它可以将生成的光谱空间特征从图像空间映射到潜在空间，并产生一个耦合组件来正则化生成的样本。从本质上讲，我们把HSI作为一个高维流形嵌入在一个潜在的空间。因此，GAN模型的优化被转换为学习潜在空间中的HR HSI样本的分布的问题，使得生成的SR HSI的分布更接近它们的原始HR对应物的分布。我们已经进行了实验评估的模型性能的SR和它的能力，在减轻模式崩溃。所提出的方法已经基于具有不同传感器的两个真实的HSI数据集（即，AVIRIS和UHD-185）对于各种放大因子（即，x2，x4，和x8）和增加的噪声电平（即，无穷大、40和80 dB），并与现有技术的SR模型（即，高光谱耦合网络（HyCoNet）、低张量训练秩（LTTR）、频带注意GAN（BAGAN）、SR-GAN和WGAN）。实验结果表明，该模型在SR质量、鲁棒性和减轻模式崩溃方面优于竞争对手。所提出的方法是能够捕捉光谱和空间的细节，并产生更忠实的样本比它的竞争对手。它也被发现，该模型是更强大的噪声和不太敏感的放大因子，并已被证明是有效的，在提高收敛的发生器和频谱空间保真度的SR HSIs。
Realistic hyperspectral image (HSI) super-resolution ( SR) techniques aim to generate a high-resolution (HR) HSI with higher spectral and spatial fidelity from its low-resolution (LR) counterpart. The generative adversarial network (GAN) has proven to be an effective deep learning framework for image SR. However, the optimization process of existing GAN-based models frequently suffers from the problem of mode collapse, leading to the limited capacity of spectral-spatial invariant reconstruction. This may cause the spectral-spatial distortion to the generated HSI, especially with a large upscaling factor. To alleviate the problem of mode collapse, this work has proposed a novel GAN model coupled with a latent encoder (LE-GAN), which can map the generated spectral-spatial features from the image space to the latent space and produce a coupling component to regularize the generated samples. Essentially, we treat an HSI as a high-dimensional manifold embedded in a latent space. Thus, the optimization of GAN models is converted to the problem of learning the distributions of HR HSI samples in the latent space, making the distributions of the generated SR HSIs closer to those of their original HR counterparts. We have conducted experimental evaluations on the model performance of SR and its capability in alleviating mode collapse. The proposed approach has been tested and validated based on two real HSI datasets with different sensors (i.e., AVIRIS and UHD-185) for various upscaling factors (i.e., x2, x4, and x8) and added noise levels (i.e., infinity, 40, and 80 dB) and compared with the state-of-the-art SR models (i.e., hyperspectral coupled network (HyCoNet), low tensor-train rank (LTTR), band attention GAN (BAGAN), SR-GAN, and WGAN). Experimental results show that the proposed model outperforms the competitors on the SR quality, robustness, and alleviation of mode collapse. The proposed approach is able to capture spectral and spatial details and generate more faithful samples than its competitors. It has also been found that the proposed model is more robust to noise and less sensitive to the upscaling factor and has been proven to be effective in improving the convergence of the generator and the spectral-spatial fidelity of the SR HSIs.