RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
复制标题
DOI：
--
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Atsuhiro Noguchi;T. Harada
中科院分区：
文献类型：
--
作者：
Atsuhiro Noguchi;T. Harada
从二维（2D）图像中理解三维（3D）几何形状而不需要任何标记信息，对于理解真实的世界而不产生注释成本是有希望的。本文提出了一种新的生成模型RGBD-GAN，它可以从2D图像中实现无监督的3D表示学习。所提出的方法使得相机参数条件的图像生成和深度图像生成没有任何3D注释，如相机姿势或深度。我们使用一个明确的3D一致性损失的两个RGBD图像从不同的相机参数，除了顺序GAN目标。这种损失对于任何类型的图像生成器（如DCGAN和StyleGAN）来说都是简单而有效的，需要以相机参数为条件。通过实验，我们证明了所提出的方法可以从具有各种生成器架构的2D图像中学习3D表示。
Understanding three-dimensional (3D) geometries from two-dimensional (2D) images without any labeled information is promising for understanding the real world without incurring annotation cost. We herein propose a novel generative model, RGBD-GAN, which achieves unsupervised 3D representation learning from 2D images. The proposed method enables camera parameter-conditional image generation and depth image generation without any 3D annotations, such as camera poses or depth. We use an explicit 3D consistency loss for two RGBD images generated from different camera parameters, in addition to the ordinal GAN objective. The loss is simple yet effective for any type of image generator such as DCGAN and StyleGAN to be conditioned on camera parameters. Through experiments, we demonstrated that the proposed method could learn 3D representations from 2D images with various generator architectures.