Deep Visual Geometry Machines

深度视觉几何机

• 批准号: RGPIN-2018-03788
• 负责人: Yi, KwangMoo
• 金额: $ 2.03万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=712410
• 关键词: Deep; Visual; Geometry; Machines

Visual Geometry is the process of understanding the environment and estimating the position of the observer from images. It is at the core of Autonomous Vehicles (AV) such as self-driving cars and delivery drones, and determines how well a machine can automatically navigate and interact with the world. For Augmented Reality (AR) and Mixed Reality (MR), Visual Geometry is one of the deciding factors on how realistic and stable one can mix the virtual world with the real world. Thus, advancing Visual Geometry is essential in bringing these new technologies into our daily lives. However, despite its importance, existing methods for Visual Geometry is not accurate enough to be deployed in the wild. The main reason for the limited performance is that we still rely on traditional methods when it comes to Visual Geometry, in contrast to other areas in Computer Vision, where machines now outperform humans in many tasks through the help of Deep Learning. Moreover, the current state- of-the-art Deep Learning-based methods for Visual Geometry are not performing as well as in other applications, for example recognizing images. I argue that the main reason for this contrast is that typically, for the applications where Deep Learning brought incredible success, the problem is mathematically over-determined, whereas Visual Geometry is under-determined due to the loss of depth information when projecting 3D real-world data into 2D images. To overcome this problem, I aim towards building the Deep Visual Geometry Machine, a collection of Deep Networks dedicated to each task of the Visual Geometry pipeline: extracting key information from images, finding correspondences between images, and recovering the state of the observer through correspondences. By limiting each network to a specific sub task, we can formulate the Visual Geometry problem as a collection of over-determined sub problems, which Deep Learning has shown to be good at solving. By doing so, I expect a significant improvement in the accuracy of the estimated position of the observer, as well as the mapping of the environment, possibly surpassing the humans. The proposed research will have immediate broad impact on many applications including, but not limited to, autonomous driving, delivery drones, domestic robots, augmented reality, and mixed reality. Furthermore, the proposed technology would enable machines to learn to plan and act, thus allowing even more complicated applications. This could include, for example, allowing autonomous vehicles and drones to operate in remote regions in the north of Canada. The current proposal also is expected to contribute to the training of HQP in the area of Machine Learning and Computer Vision. Currently, both areas are in high demand of HQP in Canada and world-wide.

视觉几何是理解环境和从图像中估计观察者位置的过程。它是自动驾驶汽车和送货无人机等自动驾驶汽车（AV）的核心，决定了机器自动导航和与世界互动的能力。对于增强现实（AR）和混合现实（MR）来说，视觉几何是决定虚拟世界与现实世界混合的真实感和稳定性的因素之一。因此，将这些新技术引入我们的日常生活中，推进视觉几何是必不可少的。