Generative Adversarial Frontal View to Bird View Synthesis
Generative Adversarial Frontal View to Bird View Synthesis
复制标题
生成对抗性正面视图到鸟瞰图合成
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Dahua Lin
中科院分区：
文献类型：
--
作者：
Xinge Zhu;Zhichao Yin;Jianping Shi;Hongsheng Li;Dahua Lin
环境感知是一项具有重要实用价值的工作，而鸟瞰是创造周围环境景观的重要组成部分。由于正视图和鸟瞰图之间的大间隙和严重变形，从单个正视图生成鸟瞰图图像是具有挑战性的。为了解决这个问题，我们提出了BridgeGAN，即，一种新的生成式鸟瞰图合成模型。首先，中间视图，即，单应性视图，以弥合这一巨大差距。接下来，在我们的任务中的三个视图（正视图，单应性视图和鸟瞰图）的条件下，提出了一个基于多GAN的模型来学习具有挑战性的跨视图翻译。此外，为了保证一对一的跨视图对应关系和一致的跨视图特征表示，我们的任务设计了两个一致性约束。在合成数据集上进行的大量实验表明，我们的模型生成的图像比现有方法生成的图像要好得多，具有更一致的全局外观和更清晰的细节。消融研究和讨论表明，在一些具有挑战性的情况下，它的可靠性和鲁棒性。
Environment perception is an important task with great practical value and bird view is an essential part for creating panoramas of surrounding environment. Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging. To tackle this problem, we propose the BridgeGAN, i.e., a novel generative model for bird view synthesis. First, an intermediate view, i.e., homography view, is introduced to bridge the large gap. Next, conditioned on the three views (frontal view, homography view and bird view) in our task, a multi-GAN based model is proposed to learn the challenging cross-view translation. Furthermore, to guarantee one-to-one cross-view correspondences and consistent cross-view feature representations, two consistency constraints are designed for our task. Extensive experiments conducted on a synthetic dataset have demonstrated that the images generated by our model are much better than those generated by existing methods, with more consistent global appearance and sharper details. Ablation studies and discussions show its reliability and robustness in some challenging cases.