Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild
Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild
复制标题
DOI：
10.1109/3dv50981.2020.00011
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Chaoyang Wang;Chen-Hsuan Lin;S. Lucey
中科院分区：
文献类型：
--
作者：
Chaoyang Wang;Chen-Hsuan Lin;S. Lucey
从大量的图像集合中提取出的2D地标的3D形状和姿态可以看作是一个非刚性运动结构（NRSfM）问题。然而，经典的NRSfM方法是有问题的，因为它们依赖于3D结构上的启发式先验（例如，低秩），其不能很好地扩展到大型数据集。基于学习的方法显示出比经典方法重建更广泛的3D结构集的潜力-极大地扩展了NRSfM对无时间无监督2D到3D提升的重要性。然而，这些学习方法无法有效地对透视相机进行建模或处理缺失/遮挡点-限制了它们对野外数据集的适用性。在本文中，我们提出了一种改进基于学习的NRSfM方法的通用策略[32]来解决上述问题。我们的方法Deep NRSfM++在众多大规模基准测试中实现了最先进的性能，优于经典和基于学习的2D-3D提升方法。
The recovery of 3D shape and pose from 2D landmarks stemming from a large ensemble of images can be viewed as a non-rigid structure from motion (NRSfM) problem. Classical NRSfM approaches, however, are problematic as they rely on heuristic priors on the 3D structure (e.g. low rank) that do not scale well to large datasets. Learning-based methods are showing the potential to reconstruct a much broader set of 3D structures than classical methods – dramatically expanding the importance of NRSfM to atemporal unsupervised 2D to 3D lifting. Hitherto, these learning approaches have not been able to effectively model perspective cameras or handle missing/occluded points – limiting their applicability to in-the-wild datasets. In this paper, we present a generalized strategy for improving learning-based NRSfM methods [32] to tackle the above issues. Our approach, Deep NRSfM++, achieves state-of-the-art performance across numerous large-scale benchmarks, outperforming both classical and learning-based 2D-3D lifting methods.