RI: Small: Modeling and Recognition of Landmarks and Urban Environments

该项目的目标是设计一个可扩展和健壮的系统，用于建模和表示部分地理参考图像的大型集合的时空和语义结构。具体来说，该项目针对的是著名地标和城市图像的互联网图片集。系统的功能包括三维重建、浏览、摘要、位置识别和场景分割。此外，该系统还结合了人为创建的注释，如文本和地理标签，模拟场景照明条件，并支持使用传入的图像流进行增量模型更新。该系统旨在利用社区图片集固有的冗余性来实现现有几何建模方法无法实现的鲁棒性和可扩展性。该项目的关键技术创新是一种新颖的数据结构，即有效而紧凑地捕获集合中图像之间的感知、几何和语义关系的标志性场景图。这个项目的关键方法论见解是，成功地表示和识别地标需要统计识别和几何重建方法的整合。该项目将统计推断整合到地标建模系统的所有组件中，并包括使用识别技术实现的高级语义功能的重要层。具有社会影响的潜在应用包括虚拟旅游和导航、安全和监视、文化遗产保护、沉浸式环境和电脑游戏以及电影特效。在项目过程中产生的数据集和代码将公开提供。该项目包括一个重要的教育组成部分，通过本科和研究生课程开发。

The goal of this project is to design a scalable and robust system for modeling and representing the spatiotemporal and semantic structure of large collections of partially geo-referenced imagery. Specifically, the project is aimed at Internet photo collections of images of famous landmarks and cities. The functionalities of the system include 3D reconstruction, browsing, summarization, location recognition, and scene segmentation. In addition, the system incorporates human-created annotations such as text and geo-tags, models scene illumination conditions, and supports incremental model updating using an incoming stream of images. This system is designed to take advantage of the redundancy inherent in community photo collections to achieve levels of robustness and scalability not attainable by existing geometric modeling approaches. The key technical innovation of the project is a novel data structure, the iconic scene graph that efficiently and compactly captures the perceptual, geometric, and semantic relationships between images in the collection.The key methodological insight of this project is that successful representation and recognition of landmarks requires the integration of statistical recognition and geometric reconstruction approaches. The project incorporates statistical inference into all components of the landmark modeling system, and includes a significant layer of high-level semantic functionality that is implemented using recognition techniques.Potential applications with societal impact include virtual tourism and navigation, security and surveillance, cultural heritage preservation, immersive environments and computer games, and movie special effects. Datasets and code produced in the course of the project will be made publicly available. The project includes a significant education component through undergraduate and graduate course development.