Don't Forget The Past: Recurrent Depth Estimation from Monocular Video
Don't Forget The Past: Recurrent Depth Estimation from Monocular Video
复制标题
DOI：
10.1109/lra.2020.3017478
复制
发表时间：
2020-10-01
影响因子：
5.2
通讯作者：
Van Gool, Luc
中科院分区：
文献类型：
--
作者：
Patil, Vaishakh;Van Gansbeke, Wouter;Van Gool, Luc
自动驾驶汽车需要不断更新的深度信息。到目前为止，即使该方法从视频输入开始，深度也大多是一次独立估计一帧。我们的方法生成深度图的时间序列，这使其成为在线学习方法的理想候选者。特别是，我们将三种不同类型的深度估计（监督深度预测、自监督深度预测和自监督深度完成）放入一个通用框架中。我们将相应的网络与 ConvLSTM 集成，以便可以利用跨帧深度的时空结构来产生更准确的深度估计。我们的方法很灵活。它可以仅应用于单目视频，也可以与不同类型的稀疏深度模式结合使用。我们仔细研究了循环网络的架构及其训练策略。我们是第一个成功利用循环网络进行实时自监督单目深度估计和完成的人。大量的实验表明，我们的循环方法在两种自我监督的场景中都一致且显着地优于基于图像的方法。它还优于之前三个流行组的深度估计方法。详情请参阅我们的网页(1)。
Autonomous cars need continuously updated depth information. Thus far, depth is mostly estimated independently for a single frame at a time, even if themethod starts fromvideo input. Our method produces a time series of depth maps, which makes it an ideal candidate for online learning approaches. In particular, we put three different types of depth estimation (supervised depth prediction, self-supervised depth prediction, and self-supervised depth completion) into a common framework. We integrate the corresponding networks with a ConvLSTM such that the spatiotemporal structures of depth across frames can be exploited to yield a more accurate depth estimation. Our method is flexible. It can be applied to monocular videos only or be combined with different types of sparse depth patterns. We carefully study the architecture of the recurrent network and its training strategy. We are first to successfully exploit recurrent networks for real-time self-supervised monocular depth estimation and completion. Extensive experiments show that our recurrent method outperforms its image-based counterpart consistently and significantly in both self-supervised scenarios. It also outperforms previous depth estimation methods of the three popular groups. Please refer to our webpage(1) for details.