RI: Small: Modeling and Recognition of Landmarks and Urban Environments

这个项目的目标是设计一个可扩展的和强大的系统建模和表示的时空和语义结构的大集合的部分地理参考的图像。具体而言，该项目的目的是在互联网上收集著名地标和城市的图像。该系统的功能包括三维重建，浏览，摘要，位置识别和场景分割。此外，该系统还集成了人工创建的注释（如文本和地理标签），对场景照明条件进行建模，并支持使用传入的图像流进行增量模型更新。该系统的目的是利用社区照片集固有的冗余，以实现现有的几何建模方法无法达到的鲁棒性和可扩展性水平。该项目的关键技术创新是一种新的数据结构，即图像场景图，它有效而复杂地捕捉了集合中图像之间的感知、几何和语义关系。该项目的关键方法学见解是，成功地表示和识别地标需要集成统计识别和几何重建方法。该项目将统计推理纳入地标建模系统的所有组件中，并包括一个重要的高级语义功能层，该功能层使用识别技术实现。具有社会影响的潜在应用包括虚拟旅游和导航，安全和监控，文化遗产保护，沉浸式环境和计算机游戏以及电影特效。项目过程中产生的数据集和代码将公开提供。该项目包括通过本科生和研究生课程开发的重要教育部分。

The goal of this project is to design a scalable and robust system for modeling and representing the spatiotemporal and semantic structure of large collections of partially geo-referenced imagery. Specifically, the project is aimed at Internet photo collections of images of famous landmarks and cities. The functionalities of the system include 3D reconstruction, browsing, summarization, location recognition, and scene segmentation. In addition, the system incorporates human-created annotations such as text and geo-tags, models scene illumination conditions, and supports incremental model updating using an incoming stream of images. This system is designed to take advantage of the redundancy inherent in community photo collections to achieve levels of robustness and scalability not attainable by existing geometric modeling approaches. The key technical innovation of the project is a novel data structure, the iconic scene graph that efficiently and compactly captures the perceptual, geometric, and semantic relationships between images in the collection.The key methodological insight of this project is that successful representation and recognition of landmarks requires the integration of statistical recognition and geometric reconstruction approaches. The project incorporates statistical inference into all components of the landmark modeling system, and includes a significant layer of high-level semantic functionality that is implemented using recognition techniques.Potential applications with societal impact include virtual tourism and navigation, security and surveillance, cultural heritage preservation, immersive environments and computer games, and movie special effects. Datasets and code produced in the course of the project will be made publicly available. The project includes a significant education component through undergraduate and graduate course development.