Comprehensive Human Performance Capture from Monocular Video Footage

拟议研究的目标是全面，照片逼真的模型从传统的，单目视频镜头的人类表演。我们计划设计的方法来估计的几何形状，姿势，运动，照明以及纹理和反射特性的一个人从一个单一的视频序列。此外，我们将开发算法来增强纹理细节，编辑姿势和运动，并令人信服地将模型渲染成新的场景。我们打算采用计算机视觉中基于模型和实例的方法，结合计算机图形学中的交互式校正、增强和错误隐藏渲染方法。我们的共同研究将有助于解决欠约束重建的一般挑战，照片级真实感建模和绘制。该项目旨在为创建基于电影的演员数字替身奠定有条不紊的基础，甚至可以对已经去世的演员进行数字保存。

Goal of the proposed research is to comprehensively, photo-realistically model human performances from conventional, monocular video footage. We plan to devise methods to estimate the geometry, pose, motion, illumination as well as texture and reflectance characteristics of a person from a single video sequence. Furthermore, we will develop algorithms to augment texture details, to edit pose and motion, and to convincingly render the model into novel scenes. We intend to employ model- as well as example-based approaches from Computer Vision in conjunction with interactive correction, augmentation, and error-concealing rendering methods from Computer Graphics. Our joint research shall make contributions to the general challenge of underconstrained reconstruction for photo-realistic modeling and rendering. The project intends to lay the methodical foundations for creating digital doubles of actors based on nothing more than their movies, enabling digital conservation of even already deceased actors.