Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods
Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods
复制标题
DOI：
10.3390/rs14194692
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Yidan Wang;Xuewen Zhou;Zurui Ao;Kun Xiao;Chen Yan;Q. Xin
中科院分区：
文献类型：
--
作者：
Yidan Wang;Xuewen Zhou;Zurui Ao;Kun Xiao;Chen Yan;Q. Xin
卫星平台上的传感器重访周期短，获取频繁的对地观测数据。利用卫星数据的一个限制是，由于云层污染和传感器故障，图像时间序列中缺少信息。大多数关于间隙填充和云去除的研究都是处理单个图像，现有的多时相图像恢复方法在处理大面积云污染频繁的图像时仍然存在问题。考虑到这些问题，我们提出了一种基于深度学习的方法，称为内容序列纹理生成（CSTG）网络，以生成图像的间隙填充时间序列。该方法使用深度神经网络通过考虑图像内容，纹理和时间序列来恢复具有缺失信息的遥感图像。我们设计了一个内容生成网络来初步填充缺失部分，并设计了一个序列纹理生成网络来优化间隙填充输出。我们使用不同地区的中分辨率成像光谱仪（MODIS）数据的时间序列，其中包括北美，欧洲和亚洲的各种表面特征来训练和测试所提出的模型。与参考图像相比，CSTG实现了结构相似性（SSIM）为0.953和平均绝对误差（MAE）为0.016的平均恢复时间序列的图像在人工实验。所开发的方法可以恢复图像的时间序列与详细的纹理和一般表现优于其他比较方法，特别是与大或重叠的时间序列中的丢失区域。我们的研究提供了一种有效的方法来填补遥感图像的时间序列，并强调了深度学习方法在重建遥感图像中的作用。
Sensors onboard satellite platforms with short revisiting periods acquire frequent earth observation data. One limitation to the utility of satellite-based data is missing information in the time series of images due to cloud contamination and sensor malfunction. Most studies on gap-filling and cloud removal process individual images, and existing multi-temporal image restoration methods still have problems in dealing with images that have large areas with frequent cloud contamination. Considering these issues, we proposed a deep learning-based method named content-sequence-texture generation (CSTG) network to generate gap-filled time series of images. The method uses deep neural networks to restore remote sensing images with missing information by accounting for image contents, textures and temporal sequences. We designed a content generation network to preliminarily fill in the missing parts and a sequence-texture generation network to optimize the gap-filling outputs. We used time series of Moderate-resolution Imaging Spectroradiometer (MODIS) data in different regions, which include various surface characteristics in North America, Europe and Asia to train and test the proposed model. Compared to the reference images, the CSTG achieved structural similarity (SSIM) of 0.953 and mean absolute errors (MAE) of 0.016 on average for the restored time series of images in artificial experiments. The developed method could restore time series of images with detailed texture and generally performed better than the other comparative methods, especially with large or overlapped missing areas in time series. Our study provides an available method to gap-fill time series of remote sensing images and highlights the power of the deep learning methods in reconstructing remote sensing images.