CAREER: Visual 3D Acquisition, Modeling and Rendering of the Real World

本研究课题的长期目标就是解决这些问题。拟议的项目是开发能够处理物体、人和完整环境的自动可视3D建模方法，包括复杂的形状、反射率和非刚性运动。由此产生的模型应该可以渲染新的图像，可能使用不同的照明，并进行3D测量，打开了广泛的应用。一个重要的方面将包括在采集过程中恢复相机运动和校准。更具体地说，该研究将解决以下主题：(1)消除现有方法的限制并提高灵活性和速度，(2)使用单相机和多相机方法从静态场景扩展到动态场景，(3)扩展到使用标准和/或全景相机捕获完整(沉浸式)环境，以及(4)扩展到恢复更复杂的反射特性和分离反射和照明。这项研究将朝着能够简单地捕捉真实对象、人和环境的外观，在必要时调整某些属性(例如姿势、光照)，然后无缝(重新)组合不同的真实和虚拟场景元素成为可能。教育部分是该项目不可分割的一部分。现有课程将进一步加强，并将开发更多课程，以提供完整的计算机视觉课程。重要的方面将是确保与该部内提供的计算机制图课程相结合和相辅相成，并将课程和研究紧密联系起来。为了确保更广泛的传播，我打算出版一本书，在专门的会议上组织课程，并为向更广泛的公众开放的展览提供应用结果。

The long term goal of this research problem is to tackle these problems. The proposed project is to develop automatic visual 3D modeling approaches that can deal with objects, people and complete environments, including complex shapes, reflectances and non-rigid motions. The resulting models should make it possible to render new images, possibly with different lighting, and to perform 3D measurements, opening up a wide range of applications. An important aspect will consist of also recovering camera motion and calibration during acquisition. More specifically the research will address the following topics: (1) removing limitations of existing approaches and increase flexibility and speed, (2) extension from static to dynamic scenes using single and multi-camera approaches, (3) extension towards capturing complete (immersive) environments using standard and/or panoramic cameras and (4) extensions towards recovering more complex reflectance properties and disentangling reflectance and illumination. This research will be developed towards making it possible to simply capture the appearance of real objects, people and environments, adapt some properties if necessary (e.g. pose, lighting) and then seamlessly (re)combine different real and virtual scene elements. The educational component forms an integral part of this project. Existing courses will be further enhanced and additional courses will be developed to provide a complete computer vision course offering. Important aspect will be to ensure integration and complementarity with the computer graphics courses provided within the department and to closely link courses and research. To ensure a broader dissemination, I intend to publish a book and to organize courses at specialized conferences as well as to provide application results for exhibits open to the broader public.