Learning Dense Correspondence via 3D-Guided Cycle Consistency
Learning Dense Correspondence via 3D-Guided Cycle Consistency
复制标题
DOI：
10.1109/cvpr.2016.20
复制
发表时间：
2016-04
期刊：
影响因子：
--
通讯作者：
Tinghui Zhou;Philipp Krähenbühl;Mathieu Aubry;Qi-Xing Huang;Alexei A. Efros
中科院分区：
文献类型：
--
作者：
Tinghui Zhou;Philipp Krähenbühl;Mathieu Aubry;Qi-Xing Huang;Alexei A. Efros
鉴别性深度学习方法在人类标记的基本真理丰富的问题上显示了令人印象深刻的结果，但对于难以或不可能获得标记的任务又如何呢？本文解决了一个这样的问题：在不同对象实例之间建立密集的视觉对应关系。对于这项任务，尽管我们不知道基本事实是什么，但我们知道它应该在这一类别的各种情况下保持一致。我们利用这种一致性作为监督信号来训练卷积神经网络来预测描述同一类别对象的图像对之间的跨实例对应关系。对于每一对训练图像，我们找到一个合适的3D CAD模型，并渲染两个合成视图以链接到这对图像，建立一个对应的流程4周期。我们使用渲染引擎提供的合成到合成的地面真实对应来训练ConvNet来预测合成到真实、真实到真实和真实到合成的对应，这些对应与地面真实循环一致。在测试时，不需要CAD模型。我们证明了在循环一致性监督下的端到端训练的ConvNet在通信相关任务中的性能优于最新的成对匹配方法。
Discriminative deep learning approaches have shown impressive results for problems where human-labeled ground truth is plentiful, but what about tasks where labels are difficult or impossible to obtain? This paper tackles one such problem: establishing dense visual correspondence across different object instances. For this task, although we do not know what the ground-truth is, we know it should be consistent across instances of that category. We exploit this consistency as a supervisory signal to train a convolutional neural network to predict cross-instance correspondences between pairs of images depicting objects of the same category. For each pair of training images we find an appropriate 3D CAD model and render two synthetic views to link in with the pair, establishing a correspondence flow 4-cycle. We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth. At test time, no CAD models are required. We demonstrate that our end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.