DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
复制标题
DOI：
10.1007/978-3-030-01228-1_3
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Yuliang Zou;Zelun Luo;Jia-Bin Huang
中科院分区：
文献类型：
--
作者：
Yuliang Zou;Zelun Luo;Jia-Bin Huang
我们提出了一个无监督学习框架，用于使用未标记的视频序列同时训练单视图深度预测和光流估计模型。现有的无监督方法通常利用亮度恒定性和空间平滑度先验来训练深度或流量模型。在本文中，我们建议利用几何一致性作为额外的监督信号。我们的核心思想是，对于刚性区域，我们可以使用预测的场景深度和摄像机运动，通过反向投影诱导的3D场景流来合成2D光流。刚性流（来自深度预测和相机运动）和估计流（来自光流模型）之间的差异允许我们施加跨任务一致性损失。虽然所有网络在训练期间都进行了联合优化，但它们可以在测试时独立应用。大量的实验表明，我们的深度和流量模型与最先进的无监督方法相比毫不逊色。
We present an unsupervised learning framework for simultaneously training single-view depth prediction and optical flow estimation models using unlabeled video sequences. Existing unsupervised methods often exploit brightness constancy and spatial smoothness priors to train depth or flow models. In this paper, we propose to leverage geometric consistency as additional supervisory signals. Our core idea is that for rigid regions we can use the predicted scene depth and camera motion to synthesize 2D optical flow by backprojecting the induced 3D scene flow. The discrepancy between the rigid flow (from depth prediction and camera motion) and the estimated flow (from optical flow model) allows us to impose a cross-task consistency loss. While all the networks are jointly optimized during training, they can be applied independently at test time. Extensive experiments demonstrate that our depth and flow models compare favorably with state-of-the-art unsupervised methods.