Comprehensive Human Performance Capture from Monocular Video Footage

从单眼视频片段中捕捉全面的人类表现

• 批准号: 265624298
• 负责人: Professor Dr.-Ing. Marcus Magnor
• 金额: --
• 依托单位: Institut für Informationsverarbeitung
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2015
• 资助国家: 德国
• 起止时间: 2014-12-31 至 2020-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/265624298?language=en
• 关键词: Comprehensive; Human; Performance; Capture; Monocular

Goal of the proposed research is to comprehensively, photo-realistically model human performances from conventional, monocular video footage. We plan to devise methods to estimate the geometry, pose, motion, illumination as well as texture and reflectance characteristics of a person from a single video sequence. Furthermore, we will develop algorithms to augment texture details, to edit pose and motion, and to convincingly render the model into novel scenes. We intend to employ model- as well as example-based approaches from Computer Vision in conjunction with interactive correction, augmentation, and error-concealing rendering methods from Computer Graphics. Our joint research shall make contributions to the general challenge of underconstrained reconstruction for photo-realistic modeling and rendering. The project intends to lay the methodical foundations for creating digital doubles of actors based on nothing more than their movies, enabling digital conservation of even already deceased actors.

提出的研究目标是从传统的单目视频片段中全面，逼真地模拟人类表演。我们计划设计一种方法，从单个视频序列中估计一个人的几何形状、姿势、运动、照明以及纹理和反射率特征。此外，我们将开发算法来增强纹理细节，编辑姿势和运动，并令人信服地将模型渲染成新颖的场景。我们打算采用计算机视觉中基于模型和示例的方法，并结合计算机图形学中的交互式校正、增强和错误隐藏渲染方法。我们的共同研究将有助于解决无约束重建的一般挑战，以实现逼真的建模和渲染。该项目旨在为仅根据演员的电影制作数字替身奠定系统基础，甚至可以对已经去世的演员进行数字保存。

项目成果

RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation

• DOI: 10.1109/cvpr.2019.00797
• 发表时间: 2019-02
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Bastian Wandt;B. Rosenhahn

A Kinematic Chain Space for Monocular Motion Capture

• DOI: 10.1007/978-3-030-11018-5_4
• 发表时间: 2017-02
• 期刊: ArXiv
• 作者: Bastian Wandt;H. Ackermann;B. Rosenhahn

Detailed Human Avatars from Monocular Video

• DOI: 10.1109/3dv.2018.00022
• 发表时间: 2018-08
• 期刊: 2018 International Conference on 3D Vision (3DV)
• 作者: Thiemo Alldieck;M. Magnor;Weipeng Xu;C. Theobalt;Gerard Pons-Moll

Learning to Reconstruct People in Clothing From a Single RGB Camera

• DOI: 10.1109/cvpr.2019.00127
• 发表时间: 2019-03
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Thiemo Alldieck;M. Magnor;Bharat Lal Bhatnagar;C. Theobalt;Gerard Pons-Moll

Video Based Reconstruction of 3D People Models

• DOI: 10.1109/cvpr.2018.00875
• 发表时间: 2018-03
• 期刊: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
• 作者: Thiemo Alldieck;M. Magnor;Weipeng Xu;C. Theobalt;Gerard Pons-Moll