Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients
Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients
复制标题
DOI：
10.1016/j.patcog.2020.107759
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Francisco Barranco;C. Fermüller;Y. Aloimonos;E. Ros
中科院分区：
文献类型：
--
作者：
Francisco Barranco;C. Fermüller;Y. Aloimonos;E. Ros
传统的基于图像运动的运动结构分析方法首先计算光流，然后基于极面约束求解三维运动参数，最后恢复场景的三维几何形状。然而，由于正则化引起的光流误差会导致三维运动和结构的较大误差。本文研究了在运动结构管道的早期阶段避免光流估计是否可以提高性能和一致性，并提出了一种新的基于图像梯度（法向流）的直接方法。我们的主要思想在于将正深度约束（从正常流中估计自我运动的基础）重新表述为连续的分段可微函数，这允许使用众所周知的最小化技术来解决3D运动。然后对三维运动估计进行细化，并添加基于深度的正则化来估计结构。在标准合成数据集和现实驾驶基准数据集Kitti上使用三种不同的光流算法进行的实验比较表明，除了一种情况外，该方法在所有情况下都取得了更好的精度。此外，它优于现有的基于常规流的3D运动估计技术。最后，恢复的三维几何形状也显示出非常准确。
Conventional image-motion based methods for structure from motion first compute optical flow, then solve for the 3D motion parameters based on the epipolar constraint, and finally recover the 3D geometry of the scene. However, errors in optical flow due to regularization can lead to large errors in 3D motion and structure. This paper investigates whether performance and consistency can be improved by avoiding optical flow estimation in the early stages of the structure-from-motion pipeline, and it proposes a new direct method based on image gradients (normal flow) only. Our main idea lies in a reformulation of the positive-depth constraint – the basis for estimating egomotion from normal flow – as a continuous piecewise differentiable function, which allows the use of well-known minimization techniques to solve for 3D motion. The 3D motion estimate is then refined and structure estimated adding a regularization based on depth. Experimental comparisons on standard synthetic datasets and the real-world driving benchmark dataset Kitti using three different optic flow algorithms show that the method achieves better accuracy in all but one case. Furthermore, it outperforms existing normal flow based 3D motion estimation techniques. Finally, the recovered 3D geometry is shown to be also very accurate.