Comprehensive Human Performance Capture from Monocular Video Footage

这项拟议的研究的目标是从传统的单目视频片段中全面、逼真地模拟人类的表演。我们计划设计一种方法，从单个视频序列中估计一个人的几何形状、姿势、运动、照明以及纹理和反射特性。此外，我们将开发算法来增强纹理细节，编辑姿势和运动，并令人信服地将模型渲染成新的场景。我们打算使用来自计算机视觉的基于模型和基于实例的方法，以及来自计算机图形学的交互校正、增强和错误隐藏渲染方法。我们的联合研究将为照片真实感建模和绘制中的欠约束重建这一普遍挑战做出贡献。该项目旨在为根据演员的电影创建数字替身奠定有条不紊的基础，即使是已经去世的演员也可以进行数字保存。

Goal of the proposed research is to comprehensively, photo-realistically model human performances from conventional, monocular video footage. We plan to devise methods to estimate the geometry, pose, motion, illumination as well as texture and reflectance characteristics of a person from a single video sequence. Furthermore, we will develop algorithms to augment texture details, to edit pose and motion, and to convincingly render the model into novel scenes. We intend to employ model- as well as example-based approaches from Computer Vision in conjunction with interactive correction, augmentation, and error-concealing rendering methods from Computer Graphics. Our joint research shall make contributions to the general challenge of underconstrained reconstruction for photo-realistic modeling and rendering. The project intends to lay the methodical foundations for creating digital doubles of actors based on nothing more than their movies, enabling digital conservation of even already deceased actors.