ITR: Interacting with the Visual World: Capturing, Understanding, and Predicting Appearance

这是一个为期五年的连续奖励的第一年资助。 该研究计划旨在使视觉信息处理的科学和工程取得重大进展，并解决计算视觉，计算机图形学和人机交互领域的基本问题。 今天，图像和视频剪辑在互联网上无处不在，数字视频正在改变娱乐的制作方式，远程学习被用于教育的各个方面，先进的机器视觉界面即将到来。 然而，目前，用户可以从视觉信息中受益的程度受到严重限制，因为几乎所有这些信息都是以原始形式呈现的，也就是说，它是以捕获的方式呈现的。 该项目的目标是开发所需的技术工具，以实现各种复杂的可视化数据操作。 这些工具将使用户能够自由地探索、交互和创建所呈现的物理世界的变体。例如，用户可以删除和添加对象到一个场景的图像，改变照明条件，改变表面的材料，或从一个新的视角查看场景。该项目包括一个全面的研究计划，为创建所需的科学和技术基础，使这种先进的视觉数据的操作。 一般的研究问题可以表述如下： 捕捉，理解和预测我们日常世界的外观。 在这一研究领域的成功需要一个统一的方法来解决两个领域的开放问题： 计算机视觉和计算机图形学。 研究工作将集中在五个相关领域： 感测、建模、估计、生成和评估。 有形的贡献将以传感器的形式提供新类型的视觉信息;材料，反射率和纹理的复杂模型;使用团队的新模型从最少的数据中恢复场景属性的估计算法;先进的渲染技术;以及一套全面的图像/视频数据库，用于评估该领域的工作。 结果将影响许多应用领域，包括数字成像，娱乐，虚拟环境，远程学习，电子商务，交互式产品设计，艺术修复，建筑建模，修复手术和表面检测。

This is the first year funding of a five-year continuing award. This research program is geared towards making significant advances to the science and engineering of visual information processing, and addresses fundamental problems in the fields of computational vision, computer graphics, and human-machine interactions. Today, images and video clips are ubiquitous on the internet, digital video is changing the way entertainment is produced, distance learning is used in various facets of education, and advanced visual interfaces to machines are around the corner. However, at present there are severe limits to the extent to which a user can benefit from visual information, because virtually all of this information is presented in its raw form, that is, the way it was captured. The goal of this project is to develop the technical tools needed to achieve a variety of complex manipulations of visual data. These tools will enable a user to freely explore, interact with, and create variations of the physical world being presented. For instance, a user may remove and add objects to an image of a scene, vary lighting conditions, change the materials of surfaces, or view the scene from a novel perspective.This project encompasses a comprehensive research program for creating the science and technology base required to enable such advanced manipulations of visual data. The general research problem may be stated as follows: Capturing, understanding, and predicting the appearance of our everyday world. Success in this domain of research necessitates a unified approach to open problems in two fields: computational vision and computer graphics. The research effort will focus on five pertinent areas: sensing, modeling, estimation, generation, and evaluation. The tangible contributions will be in the form of sensors that provide new types of visual information; complex models of materials, reflectances and textures; estimation algorithms that use the team's new models to recover scene properties from minimal data; advanced rendering techniques; and a set of comprehensive image/video databases for evaluation of work in this field. The results will impact numerous application domains, including digital imaging, entertainment, virtual environments, distance learning, e-commerce, interactive product design, art restoration, architectural modeling, restorative surgery, and surface inspection.