Comprehensive Human Performance Capture from Monocular Video Footage

拟议研究的目标是根据传统的单眼视频片段全面地、逼真地模拟人类表现。我们计划设计方法来估计单个视频序列中人的几何形状、姿势、运动、照明以及纹理和反射率特征。此外，我们将开发算法来增强纹理细节，编辑姿势和运动，并将模型令人信服地渲染成新颖的场景。我们打算将计算机视觉中基于模型和示例的方法与计算机图形学中的交互式校正、增强和错误隐藏渲染方法结合起来。我们的联合研究将为照片级真实感建模和渲染的欠约束重建的普遍挑战做出贡献。该项目旨在为仅根据电影创建演员的数字替身奠定系统基础，甚至能够对已故演员进行数字保护。

Goal of the proposed research is to comprehensively, photo-realistically model human performances from conventional, monocular video footage. We plan to devise methods to estimate the geometry, pose, motion, illumination as well as texture and reflectance characteristics of a person from a single video sequence. Furthermore, we will develop algorithms to augment texture details, to edit pose and motion, and to convincingly render the model into novel scenes. We intend to employ model- as well as example-based approaches from Computer Vision in conjunction with interactive correction, augmentation, and error-concealing rendering methods from Computer Graphics. Our joint research shall make contributions to the general challenge of underconstrained reconstruction for photo-realistic modeling and rendering. The project intends to lay the methodical foundations for creating digital doubles of actors based on nothing more than their movies, enabling digital conservation of even already deceased actors.