Video-based animation of people

本课题的研究目标是在保证源视频质量的前提下，解决重复使用多人视频捕获来支持交互动画的基本问题。研究将调查一个多视图视频数据库的重新采样，用于人们的交互式动画，使用户能够控制运动和视点的视频质量渲染。拟议的研究将解决视频中铰接和高度动态的非刚性结构表示的潜在问题，以便在保持视觉质量的同时进行索引，重用和操作。最终的挑战是通过重复使用捕获的视频序列来实现真实人物的视频质量渲染，允许用户像传统动画一样控制运动。最近的研究表明，通过对视频序列进行重采样，可以实现人脸和简单物体的逼真动画。该提案旨在将基于视频的动画应用于复杂的关节对象，如没有完整3D重建的人，这在以前的工作中已被证明会导致视觉质量下降。新的挑战是如何从捕捉到的不同动作的视频中合成新动作的视频序列。从视频中制作人物动画需要几个关键的进步，包括：人类姿势和运动的表示，支持生成以前看不见的运动序列；铰接对象的多视图视频表示，允许高效存储和索引；从大量示例序列中有效选择最有用的示例以合成新序列的算法；以及新的视频合成算法，用于在以前看不见的配置中铰接物体。直接从捕获的视频中制作人物动画有可能为下一代电影、电视和游戏中的视频质量内容制作提供支持技术。基于视频的人物动画将允许对捕获的视频进行强大的重用和操作（例如，为演员生成新颖的身体动作），同时在照片逼真的场景中进行无缝合成。

The goal of this research project is to solve the fundamental problem of re-using multiple-view video capture of people to support interactive animation with the quality of the source video. Research will investigate the resampling of a multiple-view video database for interactive animation of people to enable user control of movement and viewpoint with video-quality rendering. The proposed research will address the underlying problem of representation of articulated and highly dynamic non-rigid structures in video to allow indexing, reuse and manipulation whilst maintaining the visual quality. Ultimately the challenge is to enable video-quality rending of real people by reuse of captured video sequences allowing user control of movement as in conventional animation. Recent research has demonstrated photo-realistic animation of faces and simple objects by resampling video sequences.This proposal aims to take video-based animation to complex articulated objects such as people without full 3D reconstruction which has been shown to result in loss of visual quality in previous work. The new challenge is to synthesise video sequences of novel movements from captured video of different motion. Animating people from video requires several key advances, including: a representation of human posture and movement which supports the generation of sequences of previously unseen movements; a representation of multiple-view video of articulated objects allowing efficient storage and indexing; algorithms to efficiently selecting the most useful examples for the synthesis of a new sequences from a large set of example sequences; and new video synthesis algorithms for articulated objects in previously unseen configurations. Animation of people directly from captured video has the potential to provide enabling technology for next-generation, video-quality content production in film, television and games. Video-based animation of people will allow powerful re-use and manipulation of captured video (e.g., generating novel body movements for an actor), together with seamless compositing within photo realistic scenes.