DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images
DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Zhizhong Han;Chao Chen;Yu-Shen Liu;Matthias Zwicker
中科院分区：
文献类型：
--
作者：
Zhizhong Han;Chao Chen;Yu-Shen Liu;Matthias Zwicker
可微渲染器已经成功地用于从2D图像进行无监督的3D结构学习，因为它们可以在3D和2D之间架起一座桥梁。为了优化3D形状参数，当前的渲染器依赖于3D重建的渲染图像和来自相应视点的地面真实图像之间的像素级损失。因此，它们需要在每个像素处对恢复的3D结构进行内插、可见性处理以及可选地评估阴影模型。相反，我们在这里提出了一种不带渲染的可区分渲染器(DRWR)，它省略了这些步骤。DRWR只依赖于一个简单但有效的损失，该损失评估重建的3D点云的投影覆盖地面真实对象轮廓的程度。具体地说，DRWR使用平滑的轮廓损失来拉动对象轮廓内的每个单独3D点的投影，并且使用结构感知排斥损失来推动落入轮廓内的每对投影彼此远离。虽然我们省略了曲面内插、可见性处理和着色，但我们的结果表明，在广泛使用的基准下，DRWR达到了最先进的精度，在定性和定量上都优于以前的方法。此外，由于DRWR的简单性，我们的培训时间显著缩短。
Differentiable renderers have been used successfully for unsupervised 3D structure learning from 2D images because they can bridge the gap between 3D and 2D. To optimize 3D shape parameters, current renderers rely on pixel-wise losses between rendered images of 3D reconstructions and ground truth images from corresponding viewpoints. Hence they require interpolation of the recovered 3D structure at each pixel, visibility handling, and optionally evaluating a shading model. In contrast, here we propose a Differentiable Renderer Without Rendering (DRWR) that omits these steps. DRWR only relies on a simple but effective loss that evaluates how well the projections of reconstructed 3D point clouds cover the ground truth object silhouette. Specifically, DRWR employs a smooth silhouette loss to pull the projection of each individual 3D point inside the object silhouette, and a structure-aware repulsion loss to push each pair of projections that fall inside the silhouette far away from each other. Although we omit surface interpolation, visibility handling, and shading, our results demonstrate that DRWR achieves state-of-the-art accuracies under widely used benchmarks, outperforming previous methods both qualitatively and quantitatively. In addition, our training times are significantly lower due to the simplicity of DRWR.