RI: Small: Building Strong Geometric Priors for Total Scene Understanding

该项目正在探索几何图像理解能力如何改变人们处理自动解释个别照片或视频的语义内容的问题的方式。通过开发算法，从整合了其他地理空间数据来源的图像中准确定位摄像头，例如建筑物的3D模型和城市地区的地图，该项目旨在显著提高计算机视觉系统理解图像内容的能力。利用强先验信息进行场景理解有着广泛的重要实际应用。在家中提供老年人护理的辅助机器人应该利用对周围环境中物体外观和位置的知识，同时适应多个时间尺度上的变化(桌子上的咖啡杯移动的频率比桌子本身高得多)。自动驾驶汽车网络可以显著受益于根据汽车和其他摄像头收集的数据流构建的动态更新的城市地图(例如，调整行为以适应临时车道关闭，从而改变典型的汽车和行人交通模式)。该项目让学生参与一系列传统学科的研究，并在加州大学欧文分校校园内吸引更多的受众，了解并将这些技术应用于新的社会和科学应用。这项研究调查了另一种方法，即场景先验(包括启示和语义属性)在3D地理空间模型坐标中表示，而不是在2D图像空间中表示。将几何背景结合到场景理解中在很大程度上是在场景几何和摄像机姿势的非常弱的先验假设下进行的。重要的是，这项研究允许直接集成非可视化数据，如地理信息系统地图。该项目正在开发适当的算法和数据集，以将这些数据与连续的图像流整合在一起，以产生一个强大的、随时间演变的(4D)场景先验，这可以提高相机姿态估计、单目几何、目标检测和语义分割的精度。

This project is exploring how capabilities for geometric image understanding can change the way people approach the problem of automatically interpreting the semantic content of individual photos or videos. By developing algorithms for accurately localizing cameras from images that integrate other sources of geo-spatial data, such as 3D models of buildings and maps of urban areas, the project aims to significantly improve the ability of computer vision systems to understand image content. Utilizing strong prior information for scene understanding has a wide range of important practical applications. An assistive robot providing elderly care in a home should leverage knowledge of the appearance and location of objects in its immediate environment while adapting to changes on multiple time scales (a coffee cup sitting on the table moves much more frequently than the table itself). A network of self-driving cars could benefit significantly from dynamically updated urban maps built from the stream of data collected by the cars and other cameras (e.g., adapting behavior to a temporary lane closure that changes typical car and pedestrian traffic patterns). The project involves students in research spanning a range of traditional disciplines and is engaging a wider audience across the UC Irvine campus in understanding and applying these technologies to novel social and scientific applications.This research investigates an alternate approach in which scene priors (including affordances and semantic attributes) are represented in 3D geo-spatial model coordinates rather than in 2D image space. Incorporating geometric context into scene understanding has largely been pursued under very weak prior assumptions on scene geometry and camera pose. Importantly, the research allows for direct integration of non-visual data such as GIS maps. The project is developing the appropriate algorithms and datasets to integrating such data along with a continual stream of images to produce a strong, temporally-evolving (4D) scene prior that can improve accuracy of camera pose estimation, monocular geometry, object detection and semantic segmentation.