CAREER: Perceptually Guided Hand Motion Synthesis

这项研究将探索利用人的感知作为算法的固有部分来自动合成虚拟角色的手部和手指动画的方法。近年来，角色动画朝着逼真的虚拟代理迈进了一大步，在身体运动捕捉、实现高真实感面部动画以及模拟布料和头发方面都有越来越好的解决方案。有了这些关键组件，创建合理的手和手指动作的需要变得重要起来，因为这些动作在交流信息方面发挥着至关重要的作用，同时还允许我们执行基本任务和处理复杂的工具。但是，与身体运动相比，手部运动的大小和复杂性不同，因此很难同时捕捉或合成两者。因此，手指运动通常仍然是手动设置动画，这是一个繁琐的过程。利用感知的发现可以创建新的算法来完成这项任务(例如，通过建议新的方法和通过辅助算法参数调整)。这项研究的最终目标是将角色动画和运动感知融合到一个跨学科的领域，产生手指和手的运动合成的新见解和方法，以及更好地理解我们是如何交流的。如果成功，这项工作将极大地推动我们设计算法，使虚拟人物栩栩如生，项目结果不仅将对计算机图形学产生广泛影响，还将对虚拟现实、机器人和假肢等应用程序产生广泛影响。该项目包括针对K-12、本科生和研究生的综合教育和外展活动。这项研究将通过将感知实验与用于手部和手指运动合成的计算机动画算法开发相结合，启动我们如何设计角色动画算法的根本转变。已经提出了几种手部和手指动画的方法和设备，每种方法和设备都有自己的缺点。其中一些方法很有希望，可以改进或合并，但选择很多，需要采取更系统的方法。这项工作将集中在两个应用：虚拟角色的数据驱动的手部运动合成，以及虚拟现实中用于交互和交流的手部运动。该计划是开发一种基于分段和姿势匹配的手部运动合成算法，基于对手指姿势和动力学相关性的感知洞察，以及基于碰撞的感知，以支持虚拟现实中的实时交互。

This research will explore ways to automatically synthesize hand and finger animation for virtual characters that exploit human perception as an inherent part of the algorithm. In recent years, character animation has taken tremendous strides towards realistic virtual agents, with increasingly better solutions for body motion capture, for achieving highly realistic facial animation, and for simulating cloth and hair. With these key components in place, the need to create plausible hand and finger motions has become important because these play a crucial role in communicating information while also allowing us to conduct basic tasks and to handle complex tools. But the differences in size and complexity of hand motions compared to body motions make it difficult to capture or synthesize both at the same time. Therefore, finger motions are typically still animated manually, which is a cumbersome process. Taking advantage of perceptual findings could enable the creation of new algorithms to accomplish this task (e.g., by suggesting new methods and by aiding in algorithm parameter adjustment). The ultimate goal of this research is to merge character animation and motion perception into an interdisciplinary field that yields new insights and approaches to finger and hand movement synthesis as well as a better understanding of how we communicate. If successful, the work will significantly advance the way we design algorithms to bring virtual characters to life, and project outcomes will have broad impact not only in computer graphics but also in applications such as virtual reality, robotics and prosthetics. The project includes integrated educational and outreach activities for K-12, undergraduate, and graduate students. This research will initiate a fundamental transformation of how we design algorithms for character animation by coupling perceptual experiments with computer animation algorithm development for hand and finger motion synthesis. Several approaches and devices have been suggested for hand and finger animation, each with their own drawbacks. Some of these approaches show promise and could be improved or combined, but the options are many and a more systematic approach is required. This work will focus on two applications: data-driven hand motion synthesis for virtual characters, and hand motions for interaction and communication in virtual reality. The plan is to develop an algorithm for hand motion synthesis based on segment and pose matching, on perceptual insights on the relevance of finger poses and dynamics, and on the perception of collisions, to support real-time interaction in virtual reality.