Data-Driven Appearance Transfer for Realistic Image Synthesis

真实感图像合成是计算机图形学的核心目标。最近的重大进展使研究人员能够以非常高的真实度对各种复杂的视觉现象进行建模。 然而，即使是最好的计算机生成的故事片也与人们可能认为的"真实的"相去甚远。 奇怪的是，问题通常不是计算机图形无法模拟日常视觉世界的物理-问题是世界本身!它太复杂、太嘈杂、太丰富、太生动，即使是最熟练、最有耐心的艺术家也无法从头开始再现。一个解决方案是使用基于图像的方法，直接捕捉世界上一切事物的视觉外观--如果可行的话。相反，这项研究工作的重点是将外观从存储的视觉数据的大型数据库转移到一个新的场景中。 原因是虽然捕捉特定场景的细节非常昂贵且耗时，但从一些相关场景获得类似信息相对容易。 有大量的视觉数据已经被捕获和提供-成千上万的网络摄像头遍布世界各地，数百万张照片放在互联网上，描绘了从撒哈拉沙漠的沙尘暴到阿拉斯加的冰川的任何东西。 每天都在增加更多的数据。我们的研究正在开发一种统一的外观转移方法。 考虑了两种广泛的场景：图像堆栈中的传输（例如网络摄像头）和单个图像传输。 在这两种情况下，主要的研究问题涉及：（1）将图像和图像堆栈分组为具有连贯材料/几何属性的区域，（2）确定场景和数据库中各个组之间的对应关系，（3）最后通过将其与输入场景的大规模结构相结合，从数据库中传输正确的外观。

Realistic image synthesis is a central goal of computer graphics. Major recent advances have allowed researchers to model a wide spectrum of complicated visual phenomena with a very high degree of realism. Yet, even the best computer-generated feature films are a far cry from what one might consider "real". Curiously, the problem is generally not with computer graphics being unable to model the physics of the everyday visual world -- the problem is with the world itself! It's just too complex, too noisy, too rich and vivid to be recreated from scratch by even the most skilled and patient artist.One solution is to use image-based methods and directly capture visual appearance of everything in the world -- if only it was feasible. Instead, this research effort centers on transferring appearance from a large database of stored visual data into a novel scene. The reason is that while capturing details of a particular scene is very expensive and time-consuming, obtaining similar information from some relevant scene is relatively easy. There is a tremendous amount of visual data that is already captured and available - thousands of webcams all over the world, millions of photographs placed on the Internet, depicting anything from sandstorms in Sahara to the glaciers in Alaska. And more data is being added every day. Our research is developing a unified approach for appearance transfer. Two broad scenarios are considered: transfer in image stacks (e.g. webcams) and single image transfer. In both cases, the major research issues involve: (1) grouping images and image stacks into regions with coherent material/geometry properties, (2) determining correspondence between various groups in the scene and the database, (3) and finally transferring the correct appearance from the database by combining it with the large-scale structure of the input scene.