Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure
Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure
复制标题
使用连体结构多尺度条件生成对抗网络的半监督遥感图像融合
DOI：
10.1109/jstars.2021.3090958
复制
发表时间：
2021-01-01
影响因子：
5.5
通讯作者：
Yao, Shaowen
中科院分区：
文献类型：
--
作者：
Jin, Xin;Huang, Shanshan;Yao, Shaowen
遥感图像融合可以生成具有高空间分辨率和光谱分辨率的综合图像。融合后的遥感影像可用于灾害监测、生态环境调查、动态监测等方面。然而，大多数现有的基于深度学习的RSIF方法需要地面实况（或参考图像）来训练模型，并且地面实况的获取是一个困难的问题。为了解决这个问题，我们提出了一种基于多尺度条件生成对抗网络的半监督RSIF方法，该方法结合了多跳连接和伪暹罗结构。该方法可以同时提取全色和多光谱图像的特征进行融合，而无需地面实况;采用的多跳连接有助于呈现图像细节。此外，我们提出了一种复合损失函数，它结合了最小二乘损失，L1损失和峰值信噪比损失来训练模型;复合损失函数可以帮助保留源图像的空间细节和光谱信息。此外，我们通过大量的实验验证了所提出的方法，结果表明，新方法可以实现出色的性能，而不依赖于地面真相。
Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semisupervised RSIF method based on the multiscale conditional generative adversarial networks by combining the multiskip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multispectral images to fuse them without a ground truth; the adopted multiskip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-to-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth.