Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction
Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction
复制标题
DOI：
10.1109/tgrs.2021.3139292
复制
发表时间：
2022-01-01
影响因子：
8.2
通讯作者：
Patel, Vishal M.
中科院分区：
文献类型：
--
作者：
Bandara, Wele Gedara Chaminda;Valanarasu, Jeya Maria Jose;Patel, Vishal M.
高光谱全色锐化是将低分辨率的高光谱图像与配准的全色图像进行合成，生成具有高光谱和空间分辨率的增强高光谱图像。最近，提出的HS泛锐化方法使用深度卷积网络（ConvNets）获得了显着的结果，其通常由三个步骤组成：1）对LR-HSI进行上采样; 2）通过ConvNet预测残差图像;以及3）通过将第一步和第二步的输出相加来获得最终的融合HSI。最近的方法利用深度图像先验（DIP）对LR-HSI进行上采样，因为它具有出色的保留空间和光谱信息的能力，而无需从大型数据集学习。然而，我们观察到，通过向传统的谱域能量函数引入额外的空间域约束，可以进一步提高上采样的HSI的质量。我们将我们的空间域约束定义为预测PAN图像和实际PAN图像之间的L-1距离。为了估计上采样HSI的PAN图像，我们还提出了一个可学习的光谱响应函数（SRF）。此外，我们注意到，上采样HSI和参考HSI之间的残差图像主要由边缘信息和非常精细的结构组成。为了准确地估计精细信息，我们提出了一种新的过完备网络，称为HyperKite，它专注于通过限制深层中的接受性增加来学习高级特征。我们在三个半合成和一个真实的HSI数据集上进行实验，以证明我们的DIP-HyperKite比最先进的泛锐化方法的优越性。我们的DIP-HyperKite的部署代码、预训练模型和最终融合输出以及用于比较的方法将在https://github.com/wgcban/DIP-HyperKite.git上公开。
Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic (PAN) image to generate an enhanced HSI with high spectral and spatial resolution. Recently, the proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: 1) upsampling the LR-HSI; 2) predicting the residual image via a ConvNet; and 3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged deep image prior (DIP) to upsample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large datasets. However, we observed that the quality of upsampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the L-1 distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the upsampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the upsampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel overcomplete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three semisynthetic and one real HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pretrained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at https://github.com/wgcban/DIP-HyperKite.git.