A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion
A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion
复制标题
DOI：
10.3390/rs13245005
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Zijun Yang;C. Diao;Bo Li
中科院分区：
文献类型：
--
作者：
Zijun Yang;C. Diao;Bo Li
高密度的时间序列遥感数据，具有详细的空间信息是非常需要的动态地球系统的监测。由于传感器的权衡，大多数遥感系统不能提供高的空间和时间分辨率的图像。时空图像融合模型提供了一个可行的解决方案，以产生这样一种类型的卫星图像，但现有的融合方法是有限的，在预测快速和/或短暂的物候变化。此外，一个系统的方法来评估和了解不同程度的时间物候变化影响融合结果是缺乏时空融合研究。本研究的目的是开发一种创新的混合深度学习模型，可以有效地融合各种空间和时间分辨率的卫星图像。该模型集成了两种类型的网络模型：超分辨率卷积神经网络（SRCNN）和长短期记忆（LSTM）。SRCNN可以通过恢复退化的空间细节来增强粗糙图像，而LSTM可以从时间序列图像中学习和提取时间变化模式。为了系统地评估不同程度的物候变化的影响，我们确定图像的物候过渡日期和设计三个时间的物候变化的情况下，快速，温和，最小的物候变化。混合深度学习模型以及三个基准融合模型在不同的物候变化场景中进行了评估。结果表明，当出现快速或适度的物候变化时，混合深度学习模型会产生更好的结果。它在生成具有高空间和时间分辨率的高质量时间序列数据集方面具有巨大潜力，可进一步有益于陆地系统动态研究。了解物候变化影响的创新方法将有助于我们更好地理解当前和未来融合模型的优点和缺点。
Dense time-series remote sensing data with detailed spatial information are highly desired for the monitoring of dynamic earth systems. Due to the sensor tradeoff, most remote sensing systems cannot provide images with both high spatial and temporal resolutions. Spatiotemporal image fusion models provide a feasible solution to generate such a type of satellite imagery, yet existing fusion methods are limited in predicting rapid and/or transient phenological changes. Additionally, a systematic approach to assessing and understanding how varying levels of temporal phenological changes affect fusion results is lacking in spatiotemporal fusion research. The objective of this study is to develop an innovative hybrid deep learning model that can effectively and robustly fuse the satellite imagery of various spatial and temporal resolutions. The proposed model integrates two types of network models: super-resolution convolutional neural network (SRCNN) and long short-term memory (LSTM). SRCNN can enhance the coarse images by restoring degraded spatial details, while LSTM can learn and extract the temporal changing patterns from the time-series images. To systematically assess the effects of varying levels of phenological changes, we identify image phenological transition dates and design three temporal phenological change scenarios representing rapid, moderate, and minimal phenological changes. The hybrid deep learning model, alongside three benchmark fusion models, is assessed in different scenarios of phenological changes. Results indicate the hybrid deep learning model yields significantly better results when rapid or moderate phenological changes are present. It holds great potential in generating high-quality time-series datasets of both high spatial and temporal resolutions, which can further benefit terrestrial system dynamic studies. The innovative approach to understanding phenological changes’ effect will help us better comprehend the strengths and weaknesses of current and future fusion models.