Joint Learning of 3D Shape Retrieval and Deformation
Joint Learning of 3D Shape Retrieval and Deformation
复制标题
DOI：
10.1109/cvpr46437.2021.01154
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Mikaela Angelina Uy;Vladimir G. Kim;Minhyuk Sung;Noam Aigerman;S. Chaudhuri;L. Guibas
中科院分区：
文献类型：
--
作者：
Mikaela Angelina Uy;Vladimir G. Kim;Minhyuk Sung;Noam Aigerman;S. Chaudhuri;L. Guibas
我们提出了一种新的技术，用于生成与给定目标对象图像或扫描相匹配的高质量3D模型。我们的方法是基于从3D模型数据库中检索现有形状，然后变形其部分以匹配目标形状。与以往单独关注形状检索或变形的方法不同，我们提出了一种联合学习过程，同时训练神经变形模块和检索模块使用的嵌入空间。这使得我们的网络能够学习一个变形感知的嵌入空间，从而使检索到的模型在适当的变形后更容易与目标匹配。实际上，我们使用嵌入空间来指导用于训练变形模块的形状对，从而使其能够在有意义的形状对之间学习变形。此外，我们的新型零件感知变形模块可以处理源形状上不一致和多样化的零件结构。我们不仅在我们的新框架上，而且在最近几年提出的其他最先进的神经变形模块上展示了我们的联合训练的好处。最后，我们还表明，我们的联合训练方法的性能优于各种非联合基线。
We propose a novel technique for producing high-quality 3D models that match a given target object image or scan. Our method is based on retrieving an existing shape from a database of 3D models and then deforming its parts to match the target shape. Unlike previous approaches that independently focus on either shape retrieval or deformation, we propose a joint learning procedure that simultaneously trains the neural deformation module along with the embedding space used by the retrieval module. This enables our network to learn a deformation-aware embedding space, so that retrieved models are more amenable to match the target after an appropriate deformation. In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs. Furthermore, our novel part-aware deformation module can work with inconsistent and diverse part-structures on the source shapes. We demonstrate the benefits of our joint training not only on our novel framework, but also on other state- of-the-art neural deformation modules proposed in recent years. Lastly, we also show that our jointly-trained method outperforms various non-joint baselines.