Exploring the Potential of Conditional Adversarial Networks for Optical and SAR Image Matching
Exploring the Potential of Conditional Adversarial Networks for Optical and SAR Image Matching
复制标题
DOI：
10.1109/jstars.2018.2803212
复制
发表时间：
2018-03
影响因子：
5.5
通讯作者：
N. Merkle;S. Auer;R. Müller;P. Reinartz
中科院分区：
文献类型：
--
作者：
N. Merkle;S. Auer;R. Müller;P. Reinartz
监测自然灾害或探测变化等任务可大大受益于有关某一地区或特定感兴趣物体的补充信息。通过融合高精度的共配准和地理参考数据集提供所需的信息。此外，通过从合成孔径雷达图像中提取准确可靠的地面控制点，经校准的高分辨率光学和合成孔径雷达数据能够提高光学图像的绝对地理定位精度。在本文中，我们研究了基于深度学习的匹配概念的适用性，通过匹配光学图像和SAR图像，从SAR卫星图像中生成精确和准确的GCP。为此，训练条件生成对抗网络（cGAN）以从光学图像生成类似SAR的图像补丁。为了进行训练和测试，从覆盖欧洲更大城市地区的TerraSAR-X和PRISM图像对中提取光学和SAR图像块。人工生成的补丁，然后用于改善三个已知的匹配方法的基础上归一化互相关（NCC），尺度不变的特征变换（SIFT），和二进制鲁棒不变的可缩放密钥（BRISK），这是通常不适用于光学和SAR图像的匹配的条件。结果验证了NCC，SIFT，和BRISK为基础的匹配大大受益，在匹配的准确性和精度方面，从使用人工模板。两个国家的最先进的光学和SAR匹配方法的比较显示了所提出的方法的潜力，但也揭示了一些挑战和进一步发展的必要性。
Tasks such as the monitoring of natural disasters or the detection of change highly benefit from complementary information about an area or a specific object of interest. The required information is provided by fusing high accurate coregistered and georeferenced datasets. Aligned high-resolution optical and synthetic aperture radar (SAR) data additionally enable an absolute geolocation accuracy improvement of the optical images by extracting accurate and reliable ground control points (GCPs) from the SAR images. In this paper, we investigate the applicability of a deep learning based matching concept for the generation of precise and accurate GCPs from SAR satellite images by matching optical and SAR images. To this end, conditional generative adversarial networks (cGANs) are trained to generate SAR-like image patches from optical images. For training and testing, optical and SAR image patches are extracted from TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe. The artificially generated patches are then used to improve the conditions for three known matching approaches based on normalized cross-correlation (NCC), scale-invariant feature transform (SIFT), and binary robust invariant scalable key (BRISK), which are normally not usable for the matching of optical and SAR images. The results validate that a NCC-, SIFT-, and BRISK-based matching greatly benefit, in terms of matching accuracy and precision, from the use of the artificial templates. The comparison with two state-of-the-art optical and SAR matching approaches shows the potential of the proposed method but also revealed some challenges and the necessity for further developments.