PHOENIX: Generative models and Deep Reinforcement Learning for Geospatial Computer Vision

2017年，PI和加拿大Presagis公司开始合作研究3D重建、对象分类和逼真外观建模的深度学习。该合作由NSERC CRD/DND拨款（约60万加元）资助，包括加拿大瓦尔卡地亚国防研究与发展（DRDC）的参与。该项目代号为DAEDALUS (daedalus.theICTlab.org)，运行时间为2018年至2022年。该提案以我们现有的研究成果为基础，旨在通过解决前一个项目期间确定的一些重大挑战来实现逐步改变。具体来说，在这项代号为PHOENIX的拨款中，我们将研究以下两个研究目标：(A)开发基于生成对抗网络（gan）的新型模型，用于结构一致的图像到图像翻译。诸如高分辨率卫星图像之类的地理空间数据的获取成本很高，并且特定于捕获的地理位置。协变移位是一个突出的问题，它会导致在这些数据上训练的语义分割网络失败。使用保留结构的生成模型允许将语义掩码转移到生成的合成图像上，因此，扩大了数据集的可变性并减少了偏差。(B)在复杂的、部分可观察的、大规模的环境中使用深度强化学习（RL）来完成道路提取等视觉任务。目的是研究RL在提取地理空间信息时的使用，同时确保问题仍然易于处理。本项目将培养11名HQP（2名博士，6名硕士，3名usra）。本研究有望为解决地理空间计算机视觉机器学习领域高度实际相关的复杂问题做出重大贡献。

In 2017, the PI and Presagis Inc, Canada embarked on a collaboration to investigate deep learning for 3D reconstruction, object classification, and realistic appearance modelling. The collaboration was funded by an NSERC CRD/DND grant (~CAD $600K) and included the participation of Valcartier Defence Research & Development Canada (DRDC). The project was codenamed DAEDALUS (daedalus.theICTlab.org) and ran between 2018-2022.This proposal builds upon our existing research outcomes and aims to make a step-change by addressing some of the significant challenges identified in the duration of the previous project. Specifically, in this grant -codenamed PHOENIX- we will investigate the following two research objectives: (A) the development of novel models based on Generative Adversarial Networks (GANs) for structure-consistent image-to-image translation. Geospatial data such as high-resolution satellite imagery is expensive to acquire and specific to the capture's geographical location. A covariant shift is prominent and leads to failures in semantic segmentation networks trained on such data. Using structure-preserving generative models allows the transfer of semantic masks to the generated synthetic images and, therefore, widens the datasets' variability and reduces bias. (B) use deep Reinforcement Learning (RL) in complex, partially observable, large-scale environments for vision tasks such as road extraction. The objective is to investigate the use of RL for extracting geospatial information while ensuring the problem remains tractable.This project will facilitate the training of 11 HQP(2 Ph.D., 6 Masters, 3 USRAs). This research is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of machine learning for geospatial computer vision.