GRAINS: Generative Recursive Autoencoders for INdoor Scenes
GRAINS: Generative Recursive Autoencoders for INdoor Scenes
复制标题
GRAINS：室内场景的生成递归自动编码器
DOI：
10.1145/3303766
复制
发表时间：
2019
影响因子：
6.2
通讯作者：
Zhang Hao
中科院分区：
文献类型：
--
作者：
Li Manyi;Patil Akshay Gadi;Xu Kai;Chaudhuri Siddhartha;Khan Owais;Shamir Ariel;Tu Changhe;Chen Baoquan;Cohen-Or Daniel;Zhang Hao
我们提出了一种生成神经网络，使我们能够生成大量和多样的逼真的3D室内场景，容易和高效。我们的主要观察是，室内场景结构是内在层次的。因此，我们的网络不是卷积的;它是递归神经网络，或RvNN。使用带注释的场景层次结构的数据集，我们训练变量递归自动编码器或RvNN-VAE，它在编码阶段执行场景对象分组，并在解码过程中生成场景。具体来说，一组编码器被递归地应用于基于场景中的支持、环绕和同现关系对3D对象进行分组，对有关对象的空间属性、语义和相对于层次结构中其他对象的相对定位的信息进行编码。通过训练变分自动编码器（VAE），得到的固定长度代码大致遵循高斯分布。一个新的3D场景可以生成分层的解码器从随机采样的代码从学习的分布。我们创造了我们的方法GRAINS，用于室内场景的生成递归自动编码器。我们证明了能力的GRAINS生成合理的和多样化的3D室内场景，并与现有的3D场景合成方法进行比较。我们展示了GRAINS的应用程序，包括从2D布局的3D场景建模，场景编辑和语义场景分割，通过PointNet，其性能通过我们的方法生成的大量和各种3D场景来提高。
We present agenerative neural networkthat enables us to generate plausible 3D indoor scenes in large quantities and varieties, easily and highly efficiently. Our key observation is that indoor scene structures are inherentlyhierarchical. Hence, our network is not convolutional; it is arecursiveneural network, or RvNN. Using a dataset of annotated scene hierarchies, we train avariational recursive autoencoder, or RvNN-VAE, which performs scene object grouping during its encoding phase and scene generation during decoding. Specifically, a set of encoders are recursively applied to group 3D objects based on support, surround, and co-occurrence relations in a scene, encoding information about objects’ spatial properties,semantics, andrelativepositioning with respect to other objects in the hierarchy. By training a variational autoencoder (VAE), the resulting fixed-length codes roughly follow a Gaussian distribution. A novel 3D scene can be generated hierarchically by the decoder from a randomly sampled code from the learned distribution. We coin our method GRAINS, for Generative Recursive Autoencoders for INdoor Scenes. We demonstrate the capability of GRAINS to generate plausible and diverse 3D indoor scenes and compare with existing methods for 3D scene synthesis. We show applications of GRAINS including 3D scene modeling from 2D layouts, scene editing, and semantic scene segmentation via PointNet whose performance is boosted by the large quantity and variety of 3D scenes generated by our method.