CAREER: Geometrically Coherent Image Interpretation

职业：几何相干图像解译（Alexei a . efros）摘要图像解译，即看到和理解二维图像背后的三维世界的能力，是计算机视觉问题的核心。该方案的总体目标是，给定一张图像，自动生成对所描绘场景的连贯解释。在一个层面上，这种解释应该包括机会性地识别已知物体（如人、房子、汽车、树木）和已知材料（如草、沙子、岩石、树叶），以及它们在场景中的大致位置和方向。但更重要的是，目标是捕捉整体的“场景感”，即使我们不认识它的一些组成部分。为了解决这个极其困难的任务，PI提出了一个新的框架，旨在共同建模组成场景的元素，在它们所占据的3D空间的几何环境中。因为图像中的测量量——几何形状、材料、物体和物体部分、场景类别、相机姿势等——都不是单独可靠的，所以必须以一种连贯的方式将它们放在一起考虑。拥有几何上下文表示将允许图像的所有元素在物理上“放置”在这个上下文框架中，并允许在联合优化框架中对它们和它们的3D环境进行推理。在此提案的时间框架内，PI将开发这样一个框架，该框架将允许对图像进行几何上连贯的语义解释。智力优势：该提案的核心是努力统一两种脱节的计算机视觉哲学——传统的“几何”学派处理点和面等3D量，而较新的“外观”学派则根据2D像素模式进行操作。这两种观点在这里被结合成一个连贯的框架，在这个框架中，外观和几何形状共存，相互依赖，共同产生对图像的解释。更广泛的影响：有许多重要的现实问题将受益于拟议的研究，甚至在其发展过程中。这项工作的直接应用包括：为视障人士开发导航辅助技术，为移动机器人和汽车安全开发场景感知，以及从单个图像创建图形化3D行走环境。URL: http://www.cs.cmu.edu/埃/ ImageInterpretation /

CAREER: Geometrically Coherent Image InterpretationPI: Alexei A. EfrosAbstractImage interpretation, the ability to see and understand the three-dimensional world behind a two-dimensional image, goes to the very heart of the computer vision problem. The overall objective of this proposal is, given a single image, to automatically produce a coherent interpretation of the depicted scene. On one level, suchinterpretation should include opportunistically recognizing known objects (e.g. people, houses, cars, trees) and known materials (e.g. grass, sand, rock, foliage) as well as their rough positions and orientations within the scene. But more than that, the goal is to capture the overall "sense of the scene" even if we do not recognize some of its constituent parts.To address this extremely difficult task, the PI proposes a novel framework that aims to jointly model the elements that make up a scene within the geometric context of the 3D space that they occupy. Because none of the measured quantities in the image -- geometry, materials, objects and object parts, scene classes, camera pose, etc. -- are reliable in isolation, they must all be considered together, in a coherent way. Having the geometric context representation will allow all the elements of the image to be physically "placed" within this contextual frame and will permit reasoning between them and their 3D environment in a joint optimization framework. During the timeframe of this proposal, the PI will develop such a framework which will allow a geometrically coherent semantic interpretation of a image to emerge.Intellectual Merit: At the core of the proposal is an effort to unify two disjoint computer vision philosophies -- the traditional "Geometry" school that deals with 3D quantities like points and surfaces, and the newer "Appearance" school that operates in terms of 2D pixel patterns. These two views are here combined into one coherent framework, where appearance and geometry co-exist and rely on each other to jointly produce an interpretation of an image.Broader Impact: There are a number of important real-world problems that will benefit from the proposed research even during its development. Direct applications of this work include: developing navigation assistant technology for the visually impaired, scene awareness for mobile robots and car safety, and creating graphical 3D walk-through environments from a single image. URL: http://www.cs.cmu.edu/~efros/ImageInterpretation/