Unsupervised learning of optical flow with patch consistency and occlusion estimation
Unsupervised learning of optical flow with patch consistency and occlusion estimation
复制标题
DOI：
10.1016/j.patcog.2019.107191
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Zhe Ren;Junchi Yan;Xiaokang Yang;A. Yuille;H. Zha
中科院分区：
文献类型：
--
作者：
Zhe Ren;Junchi Yan;Xiaokang Yang;A. Yuille;H. Zha
最近的工作表明，深度网络可以在没有监督的情况下进行光流估计训练。基于光度恒定性假设，这些方法大多数采用重建损失作为基于点的向后扭曲的监督。受传统基于补丁匹配的方法的启发，我们提出了基于补丁的一致性来改进普通的无监督学习方法 Ren 等人。 [1]。我们不是仅仅比较相应的像素强度，而是通过使用带有普查变换的图像块来定位对应关系，这对于光照变化和遮挡更加鲁棒。此外，设计了一种新颖的并行分支来以无监督的方式联合估计软遮挡掩模。采用掩模来加权基于补丁的一致性损失，以减轻遮挡的影响。大量实验已在 Flying Chairs、KITTI 和 MPI-Sintel 基准测试上进行。结果表明，我们的方法是有效的，并且优于使用类似 FlowNet 网络的同行无监督学习方法。
Recent works have shown that deep networks can be trained for optical flow estimation without supervision. Based on the photometric constancy assumption, most of these methods adopt the reconstruction loss as the supervision by point-based backward warping. Inspired by the traditional patch matching based approaches, we propose a patch-based consistency to improve the vanilla unsupervised learning method Ren et al. [1]. Instead of only comparing the corresponding pixel intensity, we locate the correspondence by using the image patches with census transform, which is more robust for the illumination variation and occlusion. Moreover, a novel parallel branch is devised to estimate a soft occlusion mask jointly in an unsupervised way. The mask is adopted to weight our patch-based consistency loss to alleviate the influence of the occlusion. The plenty of experiments have been implemented on Flying Chairs, KITTI and MPI-Sintel benchmarks. The results show that our method is efficient and outperforms the peer unsupervised learning methods that are using the FlowNet-liked network.