SelfVIO: Self-supervised deep monocular Visual-Inertial Odometry and depth estimation
SelfVIO: Self-supervised deep monocular Visual-Inertial Odometry and depth estimation
复制标题
DOI：
10.1016/j.neunet.2022.03.005
复制
发表时间：
2022-03-18
期刊：
影响因子：
7.8
通讯作者：
Trigoni, Niki
中科院分区：
文献类型：
--
作者：
Almalioglu, Yasin;Turan, Mehmet;Trigoni, Niki
在过去的十年里，许多有监督的深度学习方法被提出用于视觉惯性里程计(VIO)和深度图估计，这些方法需要大量的标记数据。为了克服数据限制，自监督学习已经成为一种很有前途的替代方案，它利用场景中的几何和光度一致性等约束条件。在这项研究中，我们提出了一种新的基于自监督深度学习的VIO和深度图恢复方法(SelfVIO)，该方法采用对抗性训练和自适应视觉-惯性传感器融合。SelfVIO从未标记的单目RGB图像序列和惯性测量单元(IMU)读数中学习6自由度(6-DoF)自我运动和场景深度图的联合估计。该方法能够在不需要IMU内部参数和/或IMU与摄像机之间的外部标定的情况下执行VIO。我们对所提出的框架进行了全面的定量和定性评估，并将其性能与最新的VIO、VO和可视化同时定位和地图绘制(VSLAM)方法在Kitti、EUREC和CITYSPEES数据集上进行了比较。详细的比较证明，在姿态估计和深度恢复方面，SelfVIO的性能优于最先进的VIO方法，使其成为现有文献中一种有前途的方法。(C)2022年作者(S)。由爱思唯尔有限公司出版。这是一篇在CC by License(http://creativecommons.org/licenses/by/4.0/).下的开放获取文章
In the last decade, numerous supervised deep learning approaches have been proposed for visual- inertial odometry (VIO) and depth map estimation, which require large amounts of labelled data. To overcome the data limitation, self-supervised learning has emerged as a promising alternative that exploits constraints such as geometric and photometric consistency in the scene. In this study, we present a novel self-supervised deep learning-based VIO and depth map recovery approach (SelfVIO) using adversarial training and self-adaptive visual-inertial sensor fusion. SelfVIO learns the joint estimation of 6 degrees-of-freedom (6-DoF) ego-motion and a depth map of the scene from unlabelled monocular RGB image sequences and inertial measurement unit (IMU) readings. The proposed approach is able to perform VIO without requiring IMU intrinsic parameters and/or extrinsic calibration between IMU and the camera. We provide comprehensive quantitative and qualitative evaluations of the proposed framework and compare its performance with state-of-the-art VIO, VO, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI, EuRoC and Cityscapes datasets. Detailed comparisons prove that SelfVIO outperforms state-of-the-art VIO approaches in terms of pose estimation and depth recovery, making it a promising approach among existing methods in the literature.(c) 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).