Shape Inpainting Using 3D Generative Adversarial Network and Recurrent Convolutional Networks
Shape Inpainting Using 3D Generative Adversarial Network and Recurrent Convolutional Networks
复制标题
DOI：
10.1109/iccv.2017.252
复制
发表时间：
2017-11
期刊：
影响因子：
--
通讯作者：
Weiyue Wang;Qiangui Huang;Suya You;Chao Yang;U. Neumann
中科院分区：
文献类型：
--
作者：
Weiyue Wang;Qiangui Huang;Suya You;Chao Yang;U. Neumann
卷积神经网络的最新进展在三维形状完成方面显示了良好的结果。但由于GPU内存的限制，这些方法只能产生低分辨率的输出。为了使3D模型具有语义真实性和上下文细节，我们引入了一种结合3D编解码器生成对抗网络(3D-ED-GAN)和长期递归卷积网络(LRCN)的混合框架。3DED-GAN是一种3D卷积神经网络，采用生成性对抗性范例进行训练，以低分辨率填充缺失的3D数据。LRCN采用递归神经网络结构来最大限度地减少对GPU的内存使用，并将编解码器对合并到一个长期短期记忆网络中。通过将3D模型处理为2D切片序列，LRCN将粗略的3D形状转换为更完整和更高分辨率的体积。3D-ED-GAN捕获3D形状的全局上下文结构，而LRCN定位细粒度的细节。在真实世界和合成数据上的实验结果表明，从损坏的模型重建可以得到完整的高分辨率3D对象。
Recent advances in convolutional neural networks have shown promising results in 3D shape completion. But due to GPU memory limitations, these methods can only produce low-resolution outputs. To inpaint 3D models with semantic plausibility and contextual details, we introduce a hybrid framework that combines a 3D Encoder-Decoder Generative Adversarial Network (3D-ED-GAN) and a Longterm Recurrent Convolutional Network (LRCN). The 3DED- GAN is a 3D convolutional neural network trained with a generative adversarial paradigm to fill missing 3D data in low-resolution. LRCN adopts a recurrent neural network architecture to minimize GPU memory usage and incorporates an Encoder-Decoder pair into a Long Shortterm Memory Network. By handling the 3D model as a sequence of 2D slices, LRCN transforms a coarse 3D shape into a more complete and higher resolution volume. While 3D-ED-GAN captures global contextual structure of the 3D shape, LRCN localizes the fine-grained details. Experimental results on both real-world and synthetic data show reconstructions from corrupted models result in complete and high-resolution 3D objects.