RI: Small: A Region-Based Approach to Reconstructing Urban Scenes

近年来，为大规模的城市环境重建完整和详细的三维模型已经成为许多应用的关键技术，并为不同的服务提供了一个天然的平台。尽管传统的运动结构(SFM)技术已经成熟，但它们通常不能利用城市场景丰富的全局结构，包括规则性、对称性和自对称性；城市场景中普遍存在的非常重复的形状和纹理使得传统技术的检测和匹配特征极具挑战性。这个项目开发了一种新的方法，可以从城市场景的单个或多个2D图像中推断出高精度的3D几何图形。它充分利用了城市场景丰富的全局对称性和规律性，充分利用了现代高维凸优化的强大计算工具。该方法可以直接从原始图像像素/区域中准确地恢复场景的规则3D几何形状和2D纹理，而不需要提取任何中间局部特征。研究包括开发一套实用的工具和一个完整的系统，可以显著提高大型城市场景的3D建模的效率和可扩展性，显著提高3D几何和2D外观的紧凑表示，实现在线实时渲染和可视化。该项目的研究成果可以很容易地集成到当前的计算机视觉课程中，并显著提高3D重建的效果。相关技术可用于非常广泛的商业应用，例如在线或移动视觉搜索、视觉导航、导航或监视、虚拟旅游和增强现实等。

Recently, reconstructing full and detailed 3D models for large-scale urban environments has become the crucial technology for many applications and offers a natural platform for different services. Although conventional structure from motion (SFM) techniques have been engineered to their maturity, they normally do not utilize rich global structures of urban scenes, including regularity, symmetry, and self-symmetry; and the very repetitive shapes and textures ubiquitous in urban scenes make detecting and matching features extremely challenging for the conventional techniques. This project develops a novel approach for inferring highly accurate 3D geometry from an individual or multiple 2D images of an urban scene. It takes full advantage of the rich global symmetry and regularity in urban scenes by leveraging powerful computational tools from modern high-dimensional convex optimization. The developed method can accurately recover the regular 3D geometry and 2D texture of the scene directly from the raw image pixels/regions without relying on extracting any intermediate local features. The research includes developing a set of useful tools and a full system that can significantly improve the efficiency and scalability of 3D modeling of large urban scenes and give significantly more compact representation of the 3D geometry and 2D appearance, enabling online real-time rendering and visualization.Research results from this project can be easily integrated into and significantly improve the current computer vision course on 3D reconstruction. The associated technologies can be useful for a very wide range of commercial applications such as online or mobile visual search, visual guidance, navigation, or surveillance, virtual tourism, and augmented reality etc.