Deep Visual Geometry Machines

视觉几何学是从图像中理解环境和估计观察者位置的过程。它是自动驾驶汽车和送货无人机等自动驾驶汽车（AV）的核心，并决定了机器自动导航和与世界互动的程度。对于增强现实（AR）和混合现实（MR），视觉几何是如何现实和稳定的一个可以混合虚拟世界与真实的世界的决定因素之一。因此，推进视觉几何是将这些新技术带入我们日常生活的关键。 然而，尽管它的重要性，现有的方法视觉几何是不够准确，在野外部署。性能有限的主要原因是，在视觉几何方面，我们仍然依赖传统方法，而在计算机视觉的其他领域，通过深度学习的帮助，机器现在在许多任务中表现优于人类。此外，当前最先进的基于深度学习的视觉几何方法在其他应用（例如识别图像）中表现不佳。我认为，这种对比的主要原因是，通常情况下，对于深度学习带来令人难以置信的成功的应用程序，问题在数学上是过度确定的，而视觉几何是欠确定的，因为在将3D真实世界数据投影到2D图像时会丢失深度信息。 为了克服这个问题，我的目标是构建深度视觉几何机器，这是一个深度网络的集合，专门用于视觉几何管道的每个任务：从图像中提取关键信息，找到图像之间的对应关系，并通过对应关系恢复观察者的状态。通过将每个网络限制为特定的子任务，我们可以将视觉几何问题表述为超定子问题的集合，深度学习已经证明擅长解决这些问题。通过这样做，我希望观察者估计位置的准确性以及环境的映射有显着的提高，可能超过人类。 拟议的研究将对许多应用产生直接的广泛影响，包括但不限于自动驾驶、无人机送货、家用机器人、增强现实和混合现实。此外，拟议中的技术将使机器能够学习计划和行动，从而允许更复杂的应用。例如，这可能包括允许自动驾驶汽车和无人机在加拿大北部的偏远地区运行。 目前的提案预计也将有助于HQP在机器学习和计算机视觉领域的培训。目前，这两个领域在加拿大和世界范围内对HQP的需求都很高。

Visual Geometry is the process of understanding the environment and estimating the position of the observer from images. It is at the core of Autonomous Vehicles (AV) such as self-driving cars and delivery drones, and determines how well a machine can automatically navigate and interact with the world. For Augmented Reality (AR) and Mixed Reality (MR), Visual Geometry is one of the deciding factors on how realistic and stable one can mix the virtual world with the real world. Thus, advancing Visual Geometry is essential in bringing these new technologies into our daily lives. However, despite its importance, existing methods for Visual Geometry is not accurate enough to be deployed in the wild. The main reason for the limited performance is that we still rely on traditional methods when it comes to Visual Geometry, in contrast to other areas in Computer Vision, where machines now outperform humans in many tasks through the help of Deep Learning. Moreover, the current state- of-the-art Deep Learning-based methods for Visual Geometry are not performing as well as in other applications, for example recognizing images. I argue that the main reason for this contrast is that typically, for the applications where Deep Learning brought incredible success, the problem is mathematically over-determined, whereas Visual Geometry is under-determined due to the loss of depth information when projecting 3D real-world data into 2D images. To overcome this problem, I aim towards building the Deep Visual Geometry Machine, a collection of Deep Networks dedicated to each task of the Visual Geometry pipeline: extracting key information from images, finding correspondences between images, and recovering the state of the observer through correspondences. By limiting each network to a specific sub task, we can formulate the Visual Geometry problem as a collection of over-determined sub problems, which Deep Learning has shown to be good at solving. By doing so, I expect a significant improvement in the accuracy of the estimated position of the observer, as well as the mapping of the environment, possibly surpassing the humans. The proposed research will have immediate broad impact on many applications including, but not limited to, autonomous driving, delivery drones, domestic robots, augmented reality, and mixed reality. Furthermore, the proposed technology would enable machines to learn to plan and act, thus allowing even more complicated applications. This could include, for example, allowing autonomous vehicles and drones to operate in remote regions in the north of Canada. The current proposal also is expected to contribute to the training of HQP in the area of Machine Learning and Computer Vision. Currently, both areas are in high demand of HQP in Canada and world-wide.