Neural Non-Rigid Tracking
Neural Non-Rigid Tracking
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Aljavz Bovzivc;Pablo Rodríguez Palafox;Michael Zollhofer;Angela Dai;Justus Thies;M. Nießner
中科院分区：
文献类型：
--
作者：
Aljavz Bovzivc;Pablo Rodríguez Palafox;Michael Zollhofer;Angela Dai;Justus Thies;M. Nießner
我们引入了一种新颖的、端到端可学习的、可区分的非刚性跟踪器，可以实现最先进的非刚性重建。给定非刚性移动对象的两个输入RGB-D帧，我们采用卷积神经网络来预测密集对应。这些对应关系被用作尽可能刚性（ARAP）优化问题中的约束。通过非刚性优化求解器启用梯度反向传播，我们能够以端到端的方式学习对应关系，使得它们对于非刚性跟踪任务是最佳的。此外，该公式允许以自监督的方式学习对应权重。因此，离群值和错误的对应关系被向下加权以实现鲁棒跟踪。与最先进的方法相比，我们的算法显示出更好的重建性能，同时实现了比基于深度学习的方法快85倍的对应预测。
We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction. Given two input RGB-D frames of a non-rigidly moving object, we employ a convolutional neural network to predict dense correspondences. These correspondences are used as constraints in an as-rigid-as-possible (ARAP) optimization problem. By enabling gradient back-propagation through the non-rigid optimization solver, we are able to learn correspondences in an end-to-end manner such that they are optimal for the task of non-rigid tracking. Furthermore, this formulation allows for learning correspondence weights in a self-supervised manner. Thus, outliers and wrong correspondences are down-weighted to enable robust tracking. Compared to state-of-the-art approaches, our algorithm shows improved reconstruction performance, while simultaneously achieving 85 times faster correspondence prediction than comparable deep-learning based methods.