Self-supervised Monocular Depth Estimation

自监督单目深度估计

• 批准号: 2747408
• 金额: --
• 依托单位: Aston University
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2747408
• 关键词: Self; supervised; Monocular; Depth; Estimation

"1.1 Monocular Depth EstimationA fundamental task in computer vision is understanding our and other objects'3D positioning in space. Many prior methods of depth estimation made use of stereoscopic depth, while this was an effective technique, recent advancements in Deep Neural Networks have led to the development of monocular depth estimation. Monocular depth estimation hallucinates depth from a single RGB image. Traditionally, supervised methods would have been used to determine the depth of an image per pixel as a regression task. However, recent developments use self-supervising methods that take advantage of view-synthesis between consecutive RGB frames. These methods make use of ego-pose estimation, and depth estimation to inverse warp an image frame to its consecutive image frame. This leads to the ability to not only understand the depth of a scene but to understand the camera'smotion through this scene. These methods have shown promising results and are constantly improving but there are many advancements to be made.1.2 ImprovementsIt is now a well-known issue that these methods struggle with dynamic objects, as they assume rigid motion in all scenes. Most recent methods attempt to avoid these downfalls by allowing the system to ignore the regions that cause the error in the loss function. In this project, we aim to employ these inconveniences to teach the depth and ego-pose networks more about the scene. Also, by focusing on these dynamic objects, we can track the motionof vehicles and pedestrians in a self-supervised manner, which is a vital task for autonomous driving. Furthermore, there has been a large focus on scenes that only take into consideration daylight videos with mostly sunny clear weather. Taking today's SoTA models we see that they struggle with rain, snow, fog and other more extreme weather events leading to failure cases whenever the weather is not clear. Given that in 2021 we had 149 days of rain in the UK it is a clear issue for our depth and pose models to handle rain. Modern methods attempt to handle this by fine-tuning the models for different weather events, leading to separate models for each weather condition, which demonstrates that the current depth models do not generalise well. We aim to improve this methodology while leading to much greater generalisability of the networks. As these methods are ill-posed, it is beneficial for these methods to have some form of uncertainty estimations for the depth and pose networks. Current work in this area is limited, and uncertainty estimation work needs to be developed for ego-pose and other object-pose estimations. This and further improvements for depth uncertainty would have direct industrial applications and would be a focus of this project.Penultimately, these methods struggle with texture-less regions, which has been addressed in previous work. To further these methods, we will use a more efficient loss function for view synthesis in textureless regions. Finally, we aim to handle all of these issues while leading to reductions in computation usage of the models, allowing for these methods to be applied to practical applications.1.3 Industry ApplicationMost self-driving vehicles use a sensor fusion setup of Radar, Li-DAR and stereo sensors. This setup generates accurate 3D reconstructions but leads to significant physical costs. This combination of sensors can suffer in inaccurate results as sensors may disagree. Ultimately, replacing this configuration with a simple, accurate, monocular camera setup around the vehicle would reduce the need for Li-DAR and stereo cameras for each vehicle in a fleet. While this improvement is moderate for a single vehicle, on a large fleet this would lead to compelling reductions in costs."

“1.1单目深度估计计算机视觉的一项基本任务是理解我们和其他物体在空间中的3D定位。许多先前的深度估计方法利用立体深度，虽然这是一种有效的技术，但深度神经网络的最新进展导致了单目深度估计的发展。单目深度估计从单个RGB图像产生深度幻觉。传统上，监督方法将用于确定图像的每像素深度作为回归任务。然而，最近的发展使用自我监督的方法，利用连续的RGB帧之间的视图合成。这些方法利用自我姿态估计和深度估计来将图像帧逆变形到其连续图像帧。这导致不仅能够理解场景的深度，而且能够理解相机在场景中的运动。这些方法已经显示出有希望的结果，并不断改进，但有许多进步要做。1.2提示现在是一个众所周知的问题，这些方法的斗争与动态对象，因为他们假设在所有场景中的刚性运动。最新的方法试图通过允许系统忽略导致损失函数中的误差的区域来避免这些下降。在这个项目中，我们的目标是利用这些不便来教深度和自我姿态网络更多地了解场景。此外，通过关注这些动态对象，我们可以以自我监督的方式跟踪车辆和行人的运动，这是自动驾驶的一项重要任务。此外，已经有一个很大的重点，只考虑白天的视频，主要是阳光明媚的天气场景。以今天的SoTA模型为例，我们看到它们在雨、雪、雾和其他更极端的天气事件中挣扎，导致天气不明朗时出现故障。鉴于2021年英国有149天的降雨，我们的深度和姿势模型处理降雨显然是一个问题。现代方法试图通过针对不同天气事件微调模型来处理这一问题，从而为每种天气条件建立单独的模型，这表明当前的深度模型不能很好地推广。我们的目标是改进这种方法，同时导致更大的网络的普遍性。由于这些方法是不适定的，因此对于这些方法来说，具有针对深度和姿态网络的某种形式的不确定性估计是有益的。目前在这方面的工作是有限的，不确定性估计工作需要开发的自我姿态和其他对象的姿态估计。深度不确定性的这种和进一步的改进将具有直接的工业应用，并且将是本项目的焦点。为了进一步这些方法，我们将使用一个更有效的损失函数在无纹理区域的视图合成。最后，我们的目标是处理所有这些问题，同时减少模型的计算量，使这些方法能够应用于实际应用。1.3行业应用大多数自动驾驶车辆使用雷达，Li-DAR和立体声传感器的传感器融合设置。这种设置生成精确的3D重建，但导致显著的物理成本。传感器的这种组合可能遭受不准确的结果，因为传感器可能不一致。最终，用车辆周围简单、准确的单目摄像头设置取代这种配置，将减少车队中每辆车对Li-DAR和立体摄像头的需求。虽然这种改进对于单个车辆来说是适度的，但对于大型车队来说，这将导致成本的显著降低。"