PHOENIX: Generative models and Deep Reinforcement Learning for Geospatial Computer Vision

2017年，PI和加拿大Presagis Inc开始合作，研究用于3D重建、对象分类和逼真外观建模的深度学习。该合作由NSERC CRD/DND赠款（约60万加元）资助，并包括Valcartier Defence Research & Development Canada（DRDC）的参与。该项目代号为DAEDALUS（daedalus.theICTlab.org），在2018- 2022年之间运行。该提案建立在我们现有的研究成果基础上，旨在通过解决上一个项目期间确定的一些重大挑战来实现逐步改变。具体来说，在这项代号为PHOENIX的资助中，我们将研究以下两个研究目标：（A）开发基于生成对抗网络（GAN）的新型模型，用于结构一致的图像到图像翻译。获取高分辨率卫星图像等地理空间数据的成本很高，而且这些数据只针对捕获的地理位置。协变移位是突出的，并导致在这些数据上训练的语义分割网络失败。使用结构保持生成模型允许将语义掩码转移到生成的合成图像，因此，扩大了数据集的可变性并减少了偏差。(B)在复杂的、部分可观察的大规模环境中使用深度强化学习（RL）来完成道路提取等视觉任务。目标是调查使用RL提取地理空间信息，同时确保问题仍然易于处理。该项目将促进11名HQP（2名博士，6名硕士，3名USRA）。这项研究有望为解决与地理空间计算机视觉机器学习领域具有高度实际相关性的复杂问题做出重大贡献。

In 2017, the PI and Presagis Inc, Canada embarked on a collaboration to investigate deep learning for 3D reconstruction, object classification, and realistic appearance modelling. The collaboration was funded by an NSERC CRD/DND grant (~CAD $600K) and included the participation of Valcartier Defence Research & Development Canada (DRDC). The project was codenamed DAEDALUS (daedalus.theICTlab.org) and ran between 2018-2022.This proposal builds upon our existing research outcomes and aims to make a step-change by addressing some of the significant challenges identified in the duration of the previous project. Specifically, in this grant -codenamed PHOENIX- we will investigate the following two research objectives: (A) the development of novel models based on Generative Adversarial Networks (GANs) for structure-consistent image-to-image translation. Geospatial data such as high-resolution satellite imagery is expensive to acquire and specific to the capture's geographical location. A covariant shift is prominent and leads to failures in semantic segmentation networks trained on such data. Using structure-preserving generative models allows the transfer of semantic masks to the generated synthetic images and, therefore, widens the datasets' variability and reduces bias. (B) use deep Reinforcement Learning (RL) in complex, partially observable, large-scale environments for vision tasks such as road extraction. The objective is to investigate the use of RL for extracting geospatial information while ensuring the problem remains tractable.This project will facilitate the training of 11 HQP(2 Ph.D., 6 Masters, 3 USRAs). This research is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of machine learning for geospatial computer vision.