A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks
A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks
复制标题
DOI：
10.1109/3dv53792.2021.00140
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Yefan Zhou;Yiru Shen;Yujun Yan;Chen Feng;Yaoqing Yang
中科院分区：
文献类型：
--
作者：
Yefan Zhou;Yiru Shen;Yujun Yan;Chen Feng;Yaoqing Yang
用于单视图 3D 重建 (SVR) 的神经网络 (NN) 越来越受欢迎。最近的工作指出，对于支持向量机（SVR）来说，大多数尖端神经网络在重建不可见物体方面的性能有限，因为它们主要依赖于识别（即基于分类的方法）而不是形状重建。为了深入理解这个问题，我们系统地研究了神经网络何时以及为何更喜欢识别而不是重建，反之亦然。我们的发现表明，决定识别与重建的主要因素是训练数据的“分散”程度。因此，我们引入了分散分数（一种新的数据驱动指标）来量化这一主导因素并研究其对神经网络的影响。我们假设当训练图像更加分散并且训练形状不太分散时，神经网络会偏向于识别。通过我们对合成数据集和基准数据集的实验，我们的假设得到了支持，并且分散分数被证明是有效的。我们表明，所提出的指标是分析重建质量的主要方法，并且除了传统的重建分数之外还提供了新颖的信息。我们已经开源了我们的代码。1
Neural networks (NN) for single-view 3D reconstruction (SVR) have gained in popularity. Recent work points out that for SVR, most cutting-edge NNs have limited performance on reconstructing unseen objects because they rely primarily on recognition (i.e., classification-based methods) rather than shape reconstruction. To understand this issue in depth, we provide a systematic study on when and why NNs prefer recognition to reconstruction and vice versa. Our finding shows that a leading factor in determining recognition versus reconstruction is how “dispersed” the training data is. Thus, we introduce the dispersion score, a new data-driven metric, to quantify this leading factor and study its effect on NNs. We hypothesize that NNs are biased toward recognition when training images are more dispersed and training shapes are less dispersed. Our hypothesis is supported and the dispersion score is proved effective through our experiments on synthetic and benchmark datasets. We show that the proposed metric is a principal way to analyze reconstruction quality and provides novel information in addition to the conventional reconstruction score. We have open-sourced our code.1