Advanced Modeling of Human Movement for Computer Animation

计算机动画不仅通过好莱坞大片CG(计算机图形)电影和视频游戏影响了世界，还通过虚拟现实应用程序在模拟环境中使用虚拟工具培训工程师和医生。尽管计算机动画总体上取得了巨大的成功，但3D角色动画仍然主要依赖于动作捕捉，因此仍然是劳动密集型和成本高昂的。《阿凡达》的制作花费了数十家视觉特效公司和数千名数字艺术家数年的时间来捕捉真人演员的表演，然后在需要修改的情况下对捕捉到的动画进行手动编辑。我们认为角色动画是当前计算机图形学和创造性内容创作的瓶颈。*我建议用一种整体的方法来模拟人体运动，首先加深我们对人体运动的理解，然后让CG专业人士和新手都能轻松地创建更丰富、更逼真的人体动作。例如，给出一个5分钟的人体运动捕捉数据，我们是否可以自动装备一个数字化身来移动和完成任务，拥有数据中看到的所有技能？为了实现这一总体目标，我们提出了三个具体的短期目标：(1)深入分析和理解运动捕捉数据集；(2)学习交互速率下的个人技能级别的控制模型；(3)开发一个技能管理器，为数字化身配备人类级别的运动智能。我计划开发新的算法，将基于模型的控制方法与最近基于深度学习的无模型方法相结合。我还打算研究多种学习方法。一般来说，与计算机视觉相比，深度学习如何改变计算机动画中的问题解决并不明显，因为这两个领域之间存在显著差异。我希望通过我在未来五年的研究为回答这个问题做出贡献。人体运动的建模是计算机动画、计算机视觉、生物力学和类人机器人学中的一个基本问题。因此，许多非CG应用也可以受益于人类运动的高级建模，例如视频监控中的活动识别、受伤患者的诊断和康复以及类人机器人控制和交互。因此，尽管这项研究是在计算机动画的背景下进行的，但我们预计这项研究在计算机科学、工程和医学领域将产生更广泛的科学影响。这项研究的结果可以直接转移到电影、游戏和VR行业的动画工具和系统中。因此，我们还预计拟议的研究将产生经济影响。此外，这项研究还可能产生社会影响，因为它将使非专家能够在我们今天拍摄照片和视频时以简单直观的方式创建动画内容。

Computer animation has influenced the world not only through Hollywood blockbuster CG (Computer Graphics) movies and video games, but also via virtual reality applications that train engineers and doctors with virtual tools in simulated environments. Despite the huge success of computer animation in general, 3D Character Animation still relies mostly on motion capture and thus remains labor-intensive and costly. The making of Avatar took dozens of visual effects companies and thousands of digital artists years on performance capture of real human actors, and manual editing of the captured animations afterwards if any changes are needed. We deem character animation the current bottleneck of computer graphics and creative content creation. ******I propose to model human movement with a holistic approach to firstly deepen our understanding of human motion, and secondly enable CG professionals and novice users alike to create richer and more-realistic human motions with ease. For instance, given a 5-minute motion capture data of human movement, can we automatically equip a digital avatar to move about and accomplish tasks with all the skills seen in the data? To achieve this overall goal, we propose three specific short-term objectives: (1) deeply analyze and understand motion capture datasets, (2) learn control models at individual skill-level at interactive rates, and (3) develop a skill manager to equip digital avatars with human-level motion intelligence. I plan to develop novel algorithms that combine model-based control methods with recent deep-learning based model-free approaches. I also intend to investigate multiple learning methods. It is not obvious how deep learning can transform problem solving in computer animation in general, compared to what happened in computer vision, as there are significant differences between the two fields. I wish to contribute to answering this question with my research for the five years to come.******Modeling of human movement is a fundamental problem in computer animation, computer vision, biomechanics, and humanoid robotics. Therefore, many non-CG applications can benefit from advanced modeling of human movement as well, such as activity recognition in video surveillance, diagnosis and rehabilitation of injured patients, and humanoid robot control and interaction. Thus although studied in a computer animation context, we expect much broader scientific impact of this research in computer science, engineering, and medical disciplines. The results of this research are directly transferable to animation tools and systems for movie, game, and VR industries. We thus also expect economic impact from the proposed research. In addition, this research will likely generate social impacts as well, as it will enable non-experts to create animation content in easy and intuitive ways as we shoot photos and videos today.