Self-supervised Spatial Reasoning on Multi-View Line Drawings
Self-supervised Spatial Reasoning on Multi-View Line Drawings
复制标题
DOI：
10.1109/cvpr52688.2022.01241
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Siyuan Xiang;Anbang Yang;Yanfei Xue;Yaoqing Yang;Chen Feng
中科院分区：
文献类型：
--
作者：
Siyuan Xiang;Anbang Yang;Yanfei Xue;Yaoqing Yang;Chen Feng
通过最先进的监督深度网络对多视图线条图进行空间推理，最近在SPARE3D数据集上显示出令人困惑的低性能[14]。基于自监督学习在大量数据可用时是有用的这一事实，我们提出了两种自监督学习方法来提高SPARE3D数据集上的视图一致性推理和相机姿态推理任务的基线性能。对于第一个任务，我们使用自监督二进制分类网络来对比任何两个相似3D对象的各种视图之间的线条画差异，使训练后的网络能够有效地学习3D对象的细节敏感但视图不变的线条画表示。对于第二种类型的任务，我们提出了一个自监督的多类分类框架来训练模型，以选择正确的相应视图，从中渲染线条画。我们的方法甚至有助于下游任务与看不见的相机姿势。实验表明，我们的方法可以显着提高SPARE3D的基线性能，而一些流行的自监督学习方法不能。
Spatial reasoning on multi-view line drawings by state-of-the-art supervised deep networks is recently shown with puzzling low performances on the SPARE3D dataset [14]. Based on the fact that self-supervised learning is helpful when a large number of data are available, we propose two self-supervised learning approaches to improve the baseline performance for view consistency reasoning and camera pose reasoning tasks on the SPARE3D dataset. For the first task, we use a self-supervised binary classification network to contrast the line drawing differences between various views of any two similar 3D objects, enabling the trained networks to effectively learn detail-sensitive yet view-invariant line drawing representations of 3D objects. For the second type of task, we propose a self-supervised multi-class classification framework to train a model to select the correct corresponding view from which a line drawing is rendered. Our method is even helpful for the downstream tasks with unseen camera poses. Experiments show that our method could significantly increase the baseline performance in SPARE3D, while some popular self-supervised learning methods cannot.