Data-Driven Appearance Transfer for Realistic Image Synthesis

用于真实图像合成的数据驱动的外观传输

• 批准号: 0541230
• 负责人: Alexei Efros
• 金额: --
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-02-01 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0541230&HistoricalAwards=false
• 关键词: Data; Driven; Appearance; Transfer; Realistic

Realistic image synthesis is a central goal of computer graphics. Major recent advances have allowed researchers to model a wide spectrum of complicated visual phenomena with a very high degree of realism. Yet, even the best computer-generated feature films are a far cry from what one might consider "real". Curiously, the problem is generally not with computer graphics being unable to model the physics of the everyday visual world -- the problem is with the world itself! It's just too complex, too noisy, too rich and vivid to be recreated from scratch by even the most skilled and patient artist.One solution is to use image-based methods and directly capture visual appearance of everything in the world -- if only it was feasible. Instead, this research effort centers on transferring appearance from a large database of stored visual data into a novel scene. The reason is that while capturing details of a particular scene is very expensive and time-consuming, obtaining similar information from some relevant scene is relatively easy. There is a tremendous amount of visual data that is already captured and available - thousands of webcams all over the world, millions of photographs placed on the Internet, depicting anything from sandstorms in Sahara to the glaciers in Alaska. And more data is being added every day. Our research is developing a unified approach for appearance transfer. Two broad scenarios are considered: transfer in image stacks (e.g. webcams) and single image transfer. In both cases, the major research issues involve: (1) grouping images and image stacks into regions with coherent material/geometry properties, (2) determining correspondence between various groups in the scene and the database, (3) and finally transferring the correct appearance from the database by combining it with the large-scale structure of the input scene.

逼真的图像合成是计算机图形学的中心目标。最近的重大进展使研究人员能够以非常高的真实感来模拟各种复杂的视觉现象。然而，即使是最好的电脑制作的故事片也与人们所认为的“真实”相差甚远。奇怪的是，问题通常不在于计算机图形无法模拟日常视觉世界的物理——问题在于世界本身！它太复杂、太嘈杂、太丰富、太生动，即使是最熟练、最有耐心的艺术家也无法从头开始重建。一种解决方案是使用基于图像的方法，直接捕捉世界上所有事物的视觉外观——如果可行的话。相反，这项研究的重点是将存储视觉数据的大型数据库中的外观转移到一个新的场景中。原因是，虽然捕获特定场景的细节非常昂贵和耗时，但从某些相关场景中获取类似信息相对容易。有大量的视觉数据已经被捕获并可用——世界各地的数千个网络摄像头，互联网上的数百万张照片，描绘了从撒哈拉沙漠的沙尘暴到阿拉斯加的冰川的任何东西。每天都有更多的数据被添加进来。我们的研究是开发一种统一的外观转移方法。考虑了两种广泛的场景：图像堆栈传输（例如网络摄像头）和单个图像传输。在这两种情况下，主要的研究问题涉及：(1)将图像和图像堆栈分组到具有一致的材料/几何属性的区域中；(2)确定场景中各个组与数据库之间的对应关系；(3)最后结合输入场景的大规模结构从数据库中转移正确的外观。