Thin cloud removal in optical remote sensing images based on generative adversarial networks and physical model of cloud distortion
Thin cloud removal in optical remote sensing images based on generative adversarial networks and physical model of cloud distortion
复制标题
DOI：
10.1016/j.isprsjprs.2020.06.021
复制
发表时间：
2020-08
影响因子：
12.7
通讯作者：
Jun Li;Zhaocong Wu;Zhongwen Hu;Jiaqi Zhang;Mingliang Li;L. Mo;M. Molinier
中科院分区：
文献类型：
--
作者：
Jun Li;Zhaocong Wu;Zhongwen Hu;Jiaqi Zhang;Mingliang Li;L. Mo;M. Molinier
云污染是光学遥感图像中不可避免的问题。与厚云不同，薄云不会完全阻挡背景，这使得可以恢复背景信息。在本文中，我们提出了一种基于生成对抗网络（GAN）和云失真物理模型（CR-GAN-PM）的半监督方法，用于对来自不同区域的不成对图像进行薄云去除。本文还定义了一个考虑云吸收的云变形物理模型。值得注意的是，许多基于深度学习的最先进方法需要来自同一区域的成对云和无云图像，这通常是不可用的或耗时的。CR-GAN-PM有两个主要步骤：首先，基于GANs和图像分解原理，从输入的多云图像中分解无云背景和云失真层;然后，将这些层放入重新定义的云变形物理模型中，对输入的云图像进行重建。分解过程确保分解的背景层无云，重建过程确保生成的背景层与输入的多云图像相关。在Sentinel-2A图像上进行了实验，以验证所提出的CR-GAN-PM。对所有测试图像进行平均，CR-GAN-PM的SSIMS值（结构相似性指数测量）对于可见光和NIR波段分别为0.72、0.77、0.81和0.83。这些结果与基于端到端深度学习的方法相似，并且优于传统方法。输入波段数和超参数的取值对CR-GAN-PM的性能影响不大。实验结果表明，CR-GAN-PM对不同波段的薄云去除都是有效的，具有较强的鲁棒性。
Cloud contamination is an inevitable problem in optical remote sensing images. Unlike thick clouds, thin clouds do not completely block out background which makes it possible to restore background information. In this paper, we propose a semi-supervised method based on generative adversarial networks (GANs) and a physical model of cloud distortion (CR-GAN-PM) for thin cloud removal with unpaired images from different regions. A physical model of cloud distortion which takes the absorption of cloud into consideration was also defined in this paper. It is worth noting that many state-of-the-art methods based on deep learning require paired cloud and cloud-free images from the same region, which is often unavailable or time-consuming to collect. CR-GAN-PM has two main steps: first, the cloud-free background and cloud distortion layers were decomposed from an input cloudy image based on GANs and the principles of image decomposition; then, the input cloudy image was reconstructed by putting those layers into the redefined physical model of cloud distortion. The decomposition process ensured that the decomposed background layer was cloud-free and the reconstruction process ensured that generated background layer was correlated with the input cloudy image. Experiments were conducted on Sentinel-2A imagery to validate the proposed CR-GAN-PM. Averaged over all testing images, the SSIMs values (structural similarity index measurement) of CR-GAN-PM were 0.72, 0.77, 0.81 and 0.83 for visible and NIR bands respectively. Those results were similar to the end-to-end deep learning-based methods and better than traditional methods. The number of input bands and values of hyper-parameters affected little on the performance of CR-GAN-PM. Experimental results show that CR-GAN-PM is effective and robust for thin cloud removal in different bands.