HCC: Small: RUI: Parameterization and Collection of Demonstrative Gestures for Interactive Virtual Humans

该项目的研究目标是开发新的技术，基于从真实表演中获得的运动数据来生成逼真的、参数化的类人手势。这项工作提出了新颖的计算模型和交互界面，并重点研究了用于自主虚拟人交互训练和辅助应用的全身演示手势。虽然近年来手势建模取得了长足的进步，但对可以修改为表示环境中任意位置的参数化示意手势的关注较少。这类特定的手势对于许多应用程序来说是至关重要的。这类手势的典型例子包括指向并演示如何操作特定的设备或对象。为了确保系统的有效性，该项目包括指导计算手势模型开发的认知研究。为了确保达到逼真的效果，将使用从真实场景中执行演示任务的真实表演者获得的动作捕捉数据。拟议的框架还将考虑到从一组低成本可穿戴运动传感器以交互方式捕获的手势，从而能够交互地定制为广泛应用的交互式虚拟人类示威者编程所需的手势。该项目将通过两个关键贡献显著推进手势建模研究：(1)一个新的计算模型，它集成了来自运动捕捉的真实全身手势与运动修改技术的混合，以实现手势笔划时间的精确任意放置；(2)基于直接演示的新的手势建模用户界面，将真正实现无缝的以人为中心的用户界面来编程自主字符。这一方法是朝着实现能够有意义和有效地展示任务和程序的自主虚拟助理迈出的实质性一步。这个项目的交互界面组件具有变革性的潜力，使非专业用户能够使用手势编程，从而使虚拟人成为一种强大的交流媒介。该项目有可能影响对人类运动和认知建模的基本和广泛的研究问题，这是信息技术中的一个中心主题。此外，该项目还将通过制作一种独特的示意手势数据库使其他研究人员受益，该数据库将在公共项目网页上提供。它还将为学生提供独特的教育机会，并基于围绕研究主题开发或改进的新课程，为跨学科教育计划做出贡献。

The research goal of this project is to develop new techniques for producing realistic and parameterized humanlike gestures based on motion data acquired from real performances. This work proposes novel computational models and interactive interfaces, and focuses on whole-body demonstrative gestures for interactive training and assistance applications with autonomous virtual humans. Although gesture modeling has made substantial advances in recent years, less attention has been given to parameterized demonstrative gestures which can be modified to refer to arbitrary locations in the environment. This particular class of gestures is critical for a number of applications. Typical examples of such gestures include pointing to and demonstrating how to operate particular devices or objects. To ensure the system's effectiveness, this project includes cognitive studies for guiding the development of the computational gesture model. To ensure the achievement of realistic results, motion capture data obtained from real performers executing gestures for demonstration tasks within real scenarios will be employed. The proposed framework will also account for gestures captured interactively from a low-cost wearable set of motion sensors, enabling the interactive customization of gestures needed for programming interactive virtual human demonstrators for a broad range of applications. This project will significantly advance research on gesture modeling with two key contributions: (1) a novel computational model which integrates blending of realistic full-body gestures from motion capture with motion modification techniques for achieving precise arbitrary placement of the hands at the gesture stroke time, and (2) a new user interface for gesture modeling based on direct demonstrations which will truly enable a seamless human-centered user interface for programming autonomous characters. The approach constitutes a substantial step toward achieving autonomous virtual assistants which can meaningfully and effectively demonstrate tasks and procedures. The interactive interface component of this project has the transformative potential to enable gesture programming to become accessible to the non-specialized user, and therefore to enable virtual humans to become widely employed as a powerful communication medium. This project has the potential to impact the basic and broad research problem of modeling human movement and cognition, which is a central topic in information technology. This project will in addition benefit other researchers by producing a unique type of demonstrative gestures database which will be made available from a public project webpage. It will also provide unique educational opportunities for students and contribute to interdisciplinary educational programs based on new courses being developed or improved around the topics of the research.