CAREER: Geometrically Coherent Image Interpretation

职业：几何相干图像解释PI：Alexei A.图像解释，即看到和理解二维图像背后的三维世界的能力，是计算机视觉问题的核心。 该提案的总体目标是，给定单个图像，自动产生对所描绘场景的连贯解释。 在一个层面上，这样的解释应该包括机会主义地识别已知的对象（例如，人，房子，汽车，树）和已知的材料（例如，草，沙子，岩石，树叶）以及它们在场景中的大致位置和方向。 但更重要的是，我们的目标是捕捉整体的“场景感”，即使我们不认识它的一些组成部分。为了解决这个极其困难的任务，PI提出了一个新的框架，旨在共同建模的元素，构成一个场景中的几何上下文的3D空间，他们占据。 由于图像中的测量量--几何形状、材料、对象和对象部件、场景类、相机姿势等--都不是孤立可靠的，因此必须以连贯的方式将它们全部考虑在一起。 具有几何上下文表示将允许图像的所有元素被物理地“放置”在该上下文框架内，并且将允许在联合优化框架中在它们与它们的3D环境之间进行推理。 在本提案的时间范围内，PI将开发这样一个框架，该框架将允许对图像进行几何连贯的语义解释。该提案的核心是努力统一两种不相交的计算机视觉哲学-传统的“几何”学派，处理点和表面等3D量，和较新的“外观”学校，在2D像素模式方面运作。 这两种观点在这里结合成一个连贯的框架，外观和几何形状共存，并相互依赖，共同产生对图像的解释。更广泛的影响：即使在其发展过程中，也有一些重要的现实问题将受益于拟议的研究。 这项工作的直接应用包括：为视障人士开发导航辅助技术，为移动的机器人和汽车安全开发场景感知，以及从单个图像创建图形化3D漫游环境。网址：http://www.cs.cmu.edu/~efros/ImageInterpretation/

CAREER: Geometrically Coherent Image InterpretationPI: Alexei A. EfrosAbstractImage interpretation, the ability to see and understand the three-dimensional world behind a two-dimensional image, goes to the very heart of the computer vision problem. The overall objective of this proposal is, given a single image, to automatically produce a coherent interpretation of the depicted scene. On one level, suchinterpretation should include opportunistically recognizing known objects (e.g. people, houses, cars, trees) and known materials (e.g. grass, sand, rock, foliage) as well as their rough positions and orientations within the scene. But more than that, the goal is to capture the overall "sense of the scene" even if we do not recognize some of its constituent parts.To address this extremely difficult task, the PI proposes a novel framework that aims to jointly model the elements that make up a scene within the geometric context of the 3D space that they occupy. Because none of the measured quantities in the image -- geometry, materials, objects and object parts, scene classes, camera pose, etc. -- are reliable in isolation, they must all be considered together, in a coherent way. Having the geometric context representation will allow all the elements of the image to be physically "placed" within this contextual frame and will permit reasoning between them and their 3D environment in a joint optimization framework. During the timeframe of this proposal, the PI will develop such a framework which will allow a geometrically coherent semantic interpretation of a image to emerge.Intellectual Merit: At the core of the proposal is an effort to unify two disjoint computer vision philosophies -- the traditional "Geometry" school that deals with 3D quantities like points and surfaces, and the newer "Appearance" school that operates in terms of 2D pixel patterns. These two views are here combined into one coherent framework, where appearance and geometry co-exist and rely on each other to jointly produce an interpretation of an image.Broader Impact: There are a number of important real-world problems that will benefit from the proposed research even during its development. Direct applications of this work include: developing navigation assistant technology for the visually impaired, scene awareness for mobile robots and car safety, and creating graphical 3D walk-through environments from a single image. URL: http://www.cs.cmu.edu/~efros/ImageInterpretation/