Video-based animation of people

本研究项目的目标是解决多视角人视频捕获的根本问题，以支持具有源视频质量的交互动画。研究将研究用于人的交互式动画的多视点视频数据库的重新采样，以使用户能够控制具有视频质量的呈现的运动和视点。建议的研究将解决在视频中表示铰接式和高度动态的非刚性结构的潜在问题，以便在保持视觉质量的同时允许索引、重复使用和操作。归根结底，挑战是通过重复使用捕获的视频序列来实现真实人物的视频质量渲染，从而允许用户像传统动画那样控制运动。最近的研究通过对视频序列的重采样展示了人脸和简单对象的照片级真实感动画，该方案旨在将基于视频的动画应用于复杂的关节对象，例如没有完全3D重建的人，这在以前的工作中已经被证明会导致视觉质量的损失。新的挑战是从捕获的不同动作的视频中合成新动作的视频序列。从视频动画人需要几个关键的进步，包括：支持生成先前未见的动作序列的人体姿势和运动的表示；允许有效存储和索引的关节对象的多视角视频的表示；从大量示例序列中高效地选择用于合成新序列的最有用的示例的算法；以及用于以前未见过的配置的关节对象的新的视频合成算法。直接从捕获的视频制作人的动画具有为电影、电视和游戏中的下一代视频质量内容制作提供支持技术的潜力。基于视频的人物动画将允许对捕获的视频进行强大的重用和操纵(例如，为演员生成新颖的身体动作)，以及在照片现实场景中无缝合成。

The goal of this research project is to solve the fundamental problem of re-using multiple-view video capture of people to support interactive animation with the quality of the source video. Research will investigate the resampling of a multiple-view video database for interactive animation of people to enable user control of movement and viewpoint with video-quality rendering. The proposed research will address the underlying problem of representation of articulated and highly dynamic non-rigid structures in video to allow indexing, reuse and manipulation whilst maintaining the visual quality. Ultimately the challenge is to enable video-quality rending of real people by reuse of captured video sequences allowing user control of movement as in conventional animation. Recent research has demonstrated photo-realistic animation of faces and simple objects by resampling video sequences.This proposal aims to take video-based animation to complex articulated objects such as people without full 3D reconstruction which has been shown to result in loss of visual quality in previous work. The new challenge is to synthesise video sequences of novel movements from captured video of different motion. Animating people from video requires several key advances, including: a representation of human posture and movement which supports the generation of sequences of previously unseen movements; a representation of multiple-view video of articulated objects allowing efficient storage and indexing; algorithms to efficiently selecting the most useful examples for the synthesis of a new sequences from a large set of example sequences; and new video synthesis algorithms for articulated objects in previously unseen configurations. Animation of people directly from captured video has the potential to provide enabling technology for next-generation, video-quality content production in film, television and games. Video-based animation of people will allow powerful re-use and manipulation of captured video (e.g., generating novel body movements for an actor), together with seamless compositing within photo realistic scenes.