Example-based Inverse Rendering

“渲染”是从3D模型生成图像的计算机图形学过程。在过去的50年里，在这个问题上已经取得了巨大的进步，好莱坞电影中的照片级逼真视觉效果和视频游戏中高度引人入胜的实时图形就是明证。“逆向渲染”是从一幅或多幅图像中恢复物体或场景的形状和材质属性的计算机视觉过程，难度要大得多。这一领域的进展比正向渲染慢得多，适用于真实世界数据的方法也很少。例外的是3D形状的估计，在这种情况下，多视角立体等技术确实允许从相对不受控制的图像中恢复准确的3D模型。然而，表面的材料属性决定了光的反射方式，从而决定了它的外观。一旦这些都是已知的，就有可能预测一个物体在任何照明条件下的样子--一个所谓的“可再照明”模型。这极大地扩大了该模型可用于的应用范围。在人脸的具体情况下，可照明模型在许多领域都得到了应用，包括动画、游戏、法医成像、生物识别和视觉效果。在这个项目中，PI将访问世界领先的加州大学圣地亚哥分校视觉计算中心，合作解决人脸逆渲染问题。这个想法是使用一个面孔数据库，其中的内在反射特性已经被测量。给定要反向渲染的人脸图像，方法将是从数据库中选择与观测一致的补丁(通过正向渲染过程)，并且也是局部一致的，以确保合理的人脸外观。

"Rendering" is the computer graphics process of producing an image from a 3D model. Huge progress has been made on this problem over the past 50 years as evidenced by the photorealistic visual effects in Hollywood movies and highly engaging realtime graphics in video games. "Inverse rendering" is the much harder computer vision process of recovering shape and material properties of an object or scene from one or more images. Progress in this area has been much slower than for forward rendering and very few methods exist that are applicable to real world data. The exception is the estimation of 3D shape where techniques such as multiview stereo do allow accurate 3D models to be recovered from relatively uncontrolled imagery.However, it is the material properties of a surface that determine the way in which light is reflected and, hence, its appearance. Once these are known, it is possible to predict what an object will look like under any illumination condition - a so called "relightable" model. This dramatically expands the range of applications for which the model could be used. In the specific case of faces, relightable models find application in many areas including animation, gaming, forensic imaging, biometrics and visual effects.In this project, the PI will visit the world-leading Centre for Visual Computing at the University of California, San Diego to collaborate on an "example-based" approach to the problem of inverse rendering for faces. The idea is to use a database of faces for which the intrinsic reflectance properties have been measured. Given images of a face to inverse render, the approach will be to select patches from the database that are consistent with the observations (via a forward rendering process) and which are also locally consistent to ensure plausible face appearance.