Real-time synthesis and capture of contact rich human interactions

介绍 虚拟角色的运动合成涉及计算铰接的三维（3D）结构或骨架的旋转和关节位置。这方面的例子在现代计算机图形应用中无处不在，例如视频游戏、视觉效果和虚拟现实（VR）。近年来，在角色运动的自动合成方面已经取得了很大的进步，但是方法仍然不能达到真实的人类的技能和物理交互性的水平。众所周知，像攀爬、翻滚和灵巧操作这样的运动很难合成;跟踪和重建这些运动也同样困难。 在这些环境中，接触是一个重要因素，必须给予特别考虑。物理模型可用于提高真实性，但由于与复杂物理模拟相关的非线性和开销，鲁棒性和计算性能成为问题。因此，物理角色的自动合成方法的范围有限，因此视频游戏和VR开发人员采用它们的速度很慢。 目的 我的研究重点是虚拟角色的运动合成，其中运动需要大量的物理交互或接触。由于严格的帧率要求，将特别努力开发计算预算有限的实时应用方法。我在未来五年开发的技术将为3D角色的熟练动作合成带来新的方法，提供前所未有的交互性，可以与真实的人类的保真度相媲美。为此，我将采取三个平行但互补的办法： 1.涉及接触的快速和稳定的物理模拟 2.复杂物理相互作用的数据集和数据驱动方法 3.虚拟角色的鲁棒运动合成。 我的研究预期的结果将是交互式角色动画，运动规划，物理模拟和运动重建的创新方法。沿着发现的解决方案将立即产生实际影响。这不仅会产生游戏中角色动画的新技术，还会影响机器人模拟，VR训练和性能捕获应用。长期影响将是所有方面的进展成倍增加。 科学方法 我的一般科学方法是在适当的情况下用数据驱动的方法来增强物理模拟。这包括收集真实世界的例子来构建封装自然现象的参数化模型。这样的模型然后可以用于指导涉及物理模拟的优化。由于我的研究重点是实时应用，模型简化技术也是一个重要组成部分。挑战在于在不影响整体保真度的情况下近似物理行为。

INTRODUCTION Motion synthesis for virtual characters involves computing rotations and joint positions for an articulated three-dimensional (3D) structure, or skeleton. Examples of this are ubiquitous in modern computer graphics applications, such as video games, visual effects, and virtual reality (VR). Large strides have been made in recent years on the automatic synthesis of motion for characters, yet methods still fail to achieve the level of skill and physical interactivity of real humans. Motions such as climbing, tumbling, and dexterous manipulation are notoriously difficult to synthesize; tracking and reconstructing these motions is similarly difficult. Contact is an important factor in these settings and must be given special consideration. Physical models can be used to improve realism, but robustness and computational performance become concerns due to the non-linearities and overhead associated with complex physical simulations. Automatic synthesis methods for physical characters are therefore limited in their scope, and as such video game and VR developers have been slow to adopt them. OBJECTIVE My research focuses on motion synthesis of virtual characters where the motion demands significant amounts of physical interaction, or contact. There will be a special effort to develop methods for real-time applications with a limited computational budget due to strict framerate requirements. The techniques I develop over the next five years will give rise to new ways for skilled motion synthesis for 3D characters, offering unprecedented interactivity that rivals the fidelity of real humans. For this purpose, I will pursue three parallel, but complementary, tracks: 1. Fast and stable physical simulations involving contact; 2. Datasets and data-driven methods for complex physical interaction; 3. Robust motion synthesis for virtual characters. The anticipated results of my research will be innovative methods for interactive character animation, motion planning, physics simulation, and motion reconstruction. Solutions uncovered along the way will have immediate practical implications. This will result not only in novel technology for character animation in games, but will also impact robotics simulation, VR training, and performance capture applications. The long-term impact will be a multiplication of progress across all tracks. SCIENTIFIC APPROACH My general scientific approach is to augment physical simulations with data-driven methods where appropriate. This includes collecting real-world examples to build parameterized models that encapsulate natural phenomena. Such models may then be used to guide an optimization involving physics simulation. Since my research focuses on real-time applications, model reduction techniques are also an essential component. Challenges lie in approximating physical behaviour without compromising overall fidelity.