Deriving High Spatiotemporal Remote Sensing Images Using Deep Convolutional Network
Deriving High Spatiotemporal Remote Sensing Images Using Deep Convolutional Network
复制标题
DOI：
10.3390/rs10071066
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
Zhenyu Tan;P. Yue;L. Di;Junmei Tang
中科院分区：
文献类型：
--
作者：
Zhenyu Tan;P. Yue;L. Di;Junmei Tang
由于技术和预算的限制，遥感仪器的设计不可避免地存在一些权衡，导致难以同时获取高时空分辨率的遥感图像。针对这一问题，本文提出了一种新的数据融合模型，即深度卷积时空融合网络（DCSTFN），该模型充分利用卷积神经网络（CNN）从高时空低分辨率（HTLS）和低时空高空分辨率（LTHS）遥感图像中导出高时空分辨率图像。 DCSTFN模型由三大部分组成：HTLS图像的扩展、LTHS图像高频分量的提取以及提取特征的融合。所提出的网络的输入包括一对来自一天的 HTLS 和 LTHS 参考图像以及另一张预测日期的 HTLS 图像。卷积用于从输入中提取关键特征，反卷积用于扩展 HTLS 图像的大小。然后，从 HTLS 和 LTHS 图像中提取的特征借助考虑时间地面覆盖变化的方程进行融合。预测日的输出图像具有LTHS的空间分辨率和HTLS的时间分辨率。总体而言，DCSTFN 模型在输入和输出之间建立了复杂但直接的非线性映射。使用中分辨率成像光谱仪 (MODIS) 和陆地卫星操作陆地成像仪 (OLI) 图像进行的实验表明，所提出的基于 CNN 的方法不仅实现了最先进的精度，而且比传统的时空融合算法更加鲁棒。此外，DCSTFN 是一种更快、更省时的方法，可以与经过训练的网络进行数据融合，并且有可能应用于归档数据的批量处理。
Due to technical and budget limitations, there are inevitably some trade-offs in the design of remote sensing instruments, making it difficult to acquire high spatiotemporal resolution remote sensing images simultaneously. To address this problem, this paper proposes a new data fusion model named the deep convolutional spatiotemporal fusion network (DCSTFN), which makes full use of a convolutional neural network (CNN) to derive high spatiotemporal resolution images from remotely sensed images with high temporal but low spatial resolution (HTLS) and low temporal but high spatial resolution (LTHS). The DCSTFN model is composed of three major parts: the expansion of the HTLS images, the extraction of high frequency components from LTHS images, and the fusion of extracted features. The inputs of the proposed network include a pair of HTLS and LTHS reference images from a single day and another HTLS image on the prediction date. Convolution is used to extract key features from inputs, and deconvolution is employed to expand the size of HTLS images. The features extracted from HTLS and LTHS images are then fused with the aid of an equation that accounts for temporal ground coverage changes. The output image on the prediction day has the spatial resolution of LTHS and temporal resolution of HTLS. Overall, the DCSTFN model establishes a complex but direct non-linear mapping between the inputs and the output. Experiments with MODerate Resolution Imaging Spectroradiometer (MODIS) and Landsat Operational Land Imager (OLI) images show that the proposed CNN-based approach not only achieves state-of-the-art accuracy, but is also more robust than conventional spatiotemporal fusion algorithms. In addition, DCSTFN is a faster and less time-consuming method to perform the data fusion with the trained network, and can potentially be applied to the bulk processing of archived data.