Pan-sharpening via deep metric learning
Pan-sharpening via deep metric learning
复制标题
通过深度度量学习进行全色锐化
DOI：
10.1016/j.isprsjprs.2018.01.016
复制
发表时间：
2018
影响因子：
12.7
通讯作者：
Jiao Licheng
中科院分区：
文献类型：
--
作者：
Xing Yinghui;Wang Min;Yang Shuyuan;Jiao Licheng
基于邻域嵌入的全色锐化方法近年来受到越来越多的关注。然而，在浅层多光谱(MS)和全色(PAN)图像空间中，图像斑块并不严格遵循相似的结构，从而导致了对全色锐化的偏向。针对几何多流形邻域嵌入问题，提出了一种新的深度度量学习方法，该方法通过多个非线性深度神经网络研究面片的层次特征。首先，将来自不同卫星的下采样PAN图像划分为大量的训练图像块，然后根据其浅层几何结构对其进行粗分组。然后，将多个具有相似结构的堆叠稀疏自动编码器(SSAE)分别构造出来，并由这些分组块进行训练。在融合中，源PAN图像的图像块通过网络来提取特征，以形成深度距离度量，从而得到它们的几何标记。然后，对具有相同几何标签的面片进行分组，形成几何流形。最后，假设MS面片和PAN面片在两个不同的空间中形成相同的几何流形，将其投射到几何群上，形成几何多流形嵌入，用于高分辨率MS图像斑块的估计。在不同卫星获取的数据集上进行了一些实验。实验结果表明，无论是在视觉效果还是定量评价上，我们提出的方法都能获得比同类方法更好的融合结果。
Neighbors Embedding based pansharpening methods have received increasing interests in recent years. However, image patches do not strictly follow the similar structure in the shallow MultiSpectral (MS) and PANchromatic (PAN) image spaces, consequently leading to a bias to the pansharpening. In this paper, a new deep metric learning method is proposed to learn a refined geometric multi-manifold neighbor embedding, by exploring the hierarchical features of patches via multiple nonlinear deep neural networks. First of all, down-sampled PAN images from different satellites are divided into a large number of training image patches and are then grouped coarsely according to their shallow geometric structures. Afterwards, several Stacked Sparse AutoEncoders (SSAE) with similar structures are separately constructed and trained by these grouped patches. In the fusion, image patches of the source PAN image pass through the networks to extract features for formulating a deep distance metric and thus deriving their geometric labels. Then, patches with the same geometric labels are grouped to form geometric manifolds. Finally, the assumption that MS patches and PAN patches form the same geometric manifolds in two distinct spaces, is cast on geometric groups to formulate geometric multi-manifold embedding for estimating high resolution MS image patches. Some experiments are taken on datasets acquired by different satellites. The experimental results demonstrate that our proposed method can obtain better fusion results than its counterparts in terms of visual results and quantitative evaluations.