Example-based Inverse Rendering

点击翻译按钮获取中文摘要

"Rendering" is the computer graphics process of producing an image from a 3D model. Huge progress has been made on this problem over the past 50 years as evidenced by the photorealistic visual effects in Hollywood movies and highly engaging realtime graphics in video games. "Inverse rendering" is the much harder computer vision process of recovering shape and material properties of an object or scene from one or more images. Progress in this area has been much slower than for forward rendering and very few methods exist that are applicable to real world data. The exception is the estimation of 3D shape where techniques such as multiview stereo do allow accurate 3D models to be recovered from relatively uncontrolled imagery.However, it is the material properties of a surface that determine the way in which light is reflected and, hence, its appearance. Once these are known, it is possible to predict what an object will look like under any illumination condition - a so called "relightable" model. This dramatically expands the range of applications for which the model could be used. In the specific case of faces, relightable models find application in many areas including animation, gaming, forensic imaging, biometrics and visual effects.In this project, the PI will visit the world-leading Centre for Visual Computing at the University of California, San Diego to collaborate on an "example-based" approach to the problem of inverse rendering for faces. The idea is to use a database of faces for which the intrinsic reflectance properties have been measured. Given images of a face to inverse render, the approach will be to select patches from the database that are consistent with the observations (via a forward rendering process) and which are also locally consistent to ensure plausible face appearance.