CAREER: Visual 3D Acquisition, Modeling and Rendering of the Real World

这个研究问题的长期目标是解决这些问题。拟议的项目是开发自动视觉3D建模方法，可以处理物体，人和完整的环境，包括复杂的形状，反射和非刚性运动。由此产生的模型应该可以渲染新的图像，可能使用不同的照明，并进行3D测量，开辟广泛的应用。一个重要的方面将包括在采集过程中恢复相机运动和校准。更具体地说，研究将解决以下主题：(1)消除现有方法的局限性，提高灵活性和速度；(2)使用单相机和多相机方法从静态场景扩展到动态场景；(3)使用标准和/或全景相机扩展到捕获完整（沉浸式）环境；(4)扩展到恢复更复杂的反射率属性，解开反射率和照明的纠缠。这项研究将朝着简单地捕捉真实物体、人物和环境的外观成为可能，必要时调整一些属性（例如姿势、照明），然后无缝地（重新）组合不同的真实和虚拟场景元素。教育部分是这个项目的一个组成部分。我们会进一步加强现有的课程，并开发新的课程，以提供完整的电脑视觉课程。重要的方面是确保与系内提供的计算机图形学课程的整合和互补，并将课程和研究紧密联系起来。为了确保更广泛的传播，我打算出版一本书，在专业会议上组织课程，并为更广泛的公众开放的展览提供应用结果。

The long term goal of this research problem is to tackle these problems. The proposed project is to develop automatic visual 3D modeling approaches that can deal with objects, people and complete environments, including complex shapes, reflectances and non-rigid motions. The resulting models should make it possible to render new images, possibly with different lighting, and to perform 3D measurements, opening up a wide range of applications. An important aspect will consist of also recovering camera motion and calibration during acquisition. More specifically the research will address the following topics: (1) removing limitations of existing approaches and increase flexibility and speed, (2) extension from static to dynamic scenes using single and multi-camera approaches, (3) extension towards capturing complete (immersive) environments using standard and/or panoramic cameras and (4) extensions towards recovering more complex reflectance properties and disentangling reflectance and illumination. This research will be developed towards making it possible to simply capture the appearance of real objects, people and environments, adapt some properties if necessary (e.g. pose, lighting) and then seamlessly (re)combine different real and virtual scene elements. The educational component forms an integral part of this project. Existing courses will be further enhanced and additional courses will be developed to provide a complete computer vision course offering. Important aspect will be to ensure integration and complementarity with the computer graphics courses provided within the department and to closely link courses and research. To ensure a broader dissemination, I intend to publish a book and to organize courses at specialized conferences as well as to provide application results for exhibits open to the broader public.