PHOENIX: Generative models and Deep Reinforcement Learning for Geospatial Computer Vision

2017年，PI和加拿大Presagis Inc.开始合作，调查3D重建、对象分类和真实外观建模的深度学习。该合作由NSERC CRD/DND赠款(约60万加元)资助，并包括瓦尔卡蒂埃国防研究与发展加拿大公司(DRDC)的参与。该项目代号为Daedalus(daedalus.theICTLab.org)，在2018-2020年间运行。该提案建立在我们现有研究成果的基础上，旨在通过解决前一个项目期间确定的一些重大挑战来实现阶段性变化。具体地说，在这项代号为Phoenix的赠款中，我们将调查以下两个研究目标：(A)开发基于生成性对手网络(GANS)的新模型，用于结构一致的图像到图像的翻译。高分辨率卫星图像等地理空间数据的获取成本很高，而且特定于捕获物的地理位置。协变转移是显著的，并导致在基于这样的数据训练的语义分割网络中失败。使用保持结构的生成模型允许将语义掩码转移到生成的合成图像，因此，扩大了数据集的可变性并减少了偏差。(B)在复杂的、部分可观察的大规模环境中使用深度强化学习(RL)来执行视觉任务，如道路提取。该项目的目的是调查RL在提取地理空间信息方面的使用，同时确保问题仍然容易处理。该项目将促进11名HQP(2名博士、6名硕士、3名USRA)的培训。本研究有望为解决地理空间计算机视觉机器学习领域中具有较高实用价值的复杂问题做出实质性贡献。

In 2017, the PI and Presagis Inc, Canada embarked on a collaboration to investigate deep learning for 3D reconstruction, object classification, and realistic appearance modelling. The collaboration was funded by an NSERC CRD/DND grant (~CAD $600K) and included the participation of Valcartier Defence Research & Development Canada (DRDC). The project was codenamed DAEDALUS (daedalus.theICTlab.org) and ran between 2018-2022.This proposal builds upon our existing research outcomes and aims to make a step-change by addressing some of the significant challenges identified in the duration of the previous project. Specifically, in this grant -codenamed PHOENIX- we will investigate the following two research objectives: (A) the development of novel models based on Generative Adversarial Networks (GANs) for structure-consistent image-to-image translation. Geospatial data such as high-resolution satellite imagery is expensive to acquire and specific to the capture's geographical location. A covariant shift is prominent and leads to failures in semantic segmentation networks trained on such data. Using structure-preserving generative models allows the transfer of semantic masks to the generated synthetic images and, therefore, widens the datasets' variability and reduces bias. (B) use deep Reinforcement Learning (RL) in complex, partially observable, large-scale environments for vision tasks such as road extraction. The objective is to investigate the use of RL for extracting geospatial information while ensuring the problem remains tractable.This project will facilitate the training of 11 HQP(2 Ph.D., 6 Masters, 3 USRAs). This research is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of machine learning for geospatial computer vision.