RI: Small: Crowd-Sourcing the World: Scalable Methods for Dynamic Structure from Motion

该项目使全球范围的动态重建能够扩展到最终包含世界上所有的3D数据，任何用户都可以为其提供新的视觉数据，从而确保更完整、最新的模型，并为依赖这些数据的应用程序用户提供更好的体验。来自各种来源（消费者、工业和政府）的大量公开可用图像，以及配备摄像头的联网移动设备的普及，为构建覆盖整个世界的大规模3D模型提供了机会。这种全球3D模型对于需要实时访问环境结构的广泛应用非常宝贵，例如为视障人士提供援助，探索危险区域进行搜索和救援行动，城市规划，自动驾驶汽车和虚拟旅游。推动这项工作的关键研究问题是如何有效和准确地更新全球尺度的三维重建，以支持动态的全球模型。该研究使用运动结构（SfM）技术获取3D建模的全局背景，然后通过一种称为“全球化”的新技术将局部3D重建和其他视觉数据传播回全局模型。该项目解决了大规模SfM的一个主要缺陷：快速扩展和更新重建以促进实时使用。该项目建立在大规模图像匹配、定位和三维重建的初步工作基础上。研究小组进行广泛的数据收集和实验，以确定所开发技术的性能基准，并评估项目的进展和方法的效用。该项目正在开发一个原型移动系统，用于提供图像、浏览和更新模型，供公众使用。

This project enables global-scale dynamic reconstructions that can scale to eventually encompass all of the world's 3D data, to which any user may contribute new visual data, thereby ensuring a more complete, up-to-date model and a better experience for users of applications that rely on such data. The abundance of publicly available imagery from a variety of sources (consumer, industry, and government) and the proliferation of networked mobile devices equipped with cameras provide an opportunity to build large-scale 3D models that cover the entire world. Such global 3D models are invaluable to a wide range of applications that require real-time access to environment structure, such as providing assistance to the visually impaired, exploring dangerous areas for search-and rescue operations, urban planning, self-driving cars, and virtual tourism. The key research question driving this work is how to efficiently and accurately update a global-scale 3D reconstruction in order to support a dynamic global model. The research uses structure from motion (SfM) techniques to acquire global context for 3D modeling and then propagate local 3D reconstructions and other visual data back to the global model via a new technique called "globalization." The project addresses a major gap in large-scale SfM: rapidly extending and updating reconstructions to promote real-time use. The project is built on preliminary work in large-scale image matching, localization, and 3D reconstruction. The research team carries out extensive data collection and experimentation to benchmark the performance of the developed techniques and to assess the progress of the project and the utility of the methods. The project is developing a prototype mobile system for contributing imagery and browsing and updating models for public use once it is available.