Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images
Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images
复制标题
从单视图 RGB 图像进行弱监督的部分 3D 形状重建
DOI：
10.1111/cgf.14158
复制
发表时间：
2020
影响因子：
2.5
通讯作者：
Kai Xu
中科院分区：
文献类型：
--
作者：
Chengjie Niu;Yang Yu;Zhenwei Bian;Jun Li;Kai Xu
为了让深度学习模型真正理解2D图像以进行3D几何恢复，我们认为应该以部分感知和弱监督的方式学习单视图重建。这种模型导致对2D图像的更深刻的解释，其中涉及基于部分的解析和组装。为此，我们学习了一个深度神经网络，该网络将单视图RGB图像作为输入，并使用3D零件生成器阵列输出由3D点云表示的零件中的3D形状。特别是，我们设计了两个层次的生成对抗网络（GAN），以生成具有正确的部分形状和合理的整体结构的形状。为了实现自学网络训练，我们设计了一个可微分投影模块，沿着自投影损失，测量形状投影和输入图像之间的误差。在我们的方法中，训练数据在2D图像和具有部分分解的3D形状之间是不成对的。通过对公共数据集的定性和定量评估，我们表明我们的方法在部分单视图重建中取得了良好的性能。
In order for the deep learning models to truly understand the 2D images for 3D geometry recovery, we argue that single‐view reconstruction should be learned in a part‐aware and weakly supervised manner. Such models lead to more profound interpretation of 2D images in which part‐based parsing and assembling are involved. To this end, we learn a deep neural network which takes a single‐view RGB image as input, and outputs a 3D shape in parts represented by 3D point clouds with an array of 3D part generators. In particular, we devise two levels of generative adversarial network (GAN) to generate shapes with both correct part shape and reasonable overall structure. To enable a self‐taught network training, we devise a differentiable projection module along with a self‐projection loss measuring the error between the shape projection and the input image. The training data in our method is unpaired between the 2D images and the 3D shapes with part decomposition. Through qualitative and quantitative evaluations on public datasets, we show that our method achieves good performance in part‐wise single‐view reconstruction.