HCC: Small: RUI: Parameterization and Collection of Demonstrative Gestures for Interactive Virtual Humans

本计画的研究目标是开发新的技术，根据从真实的表演中所获得的动作数据，产生逼真且参数化的拟人手势。这项工作提出了新的计算模型和交互式界面，并专注于与自主虚拟人的交互式培训和援助应用的全身演示手势。虽然手势建模在最近几年取得了很大的进步，但较少关注可以被修改为指代环境中任意位置的参数化演示手势。这类特殊的手势对于许多应用程序来说是至关重要的。这种手势的典型示例包括指向和演示如何操作特定设备或对象。为了确保系统的有效性，该项目包括认知研究，以指导计算手势模型的开发。为了确保实现逼真的效果，将采用从真实的场景中执行演示任务的真实的表演者获得的动作捕捉数据。拟议的框架还将占互动捕捉的姿态从一个低成本的可穿戴运动传感器，使互动定制的编程交互式虚拟人演示程序的广泛应用所需的姿态。该项目将显著推进手势建模的研究，有两个关键贡献：（1）一种新颖的计算模型，其将来自运动捕捉的真实全身姿势与运动修改技术的混合集成，用于在姿势笔划时间实现手的精确任意放置，以及（2）用于基于直接演示的姿势建模的新用户界面，其将真正实现用于编程自主角色的无缝的以人为中心的用户界面。该方法是实现自主虚拟助理的重要一步，可以有效地演示任务和程序。该项目的交互式界面组件具有变革性的潜力，使手势编程成为非专业用户访问，因此，使虚拟人成为一个强大的通信媒介，广泛采用。该项目有可能影响建模人类运动和认知的基础和广泛的研究问题，这是信息技术的中心课题。此外，该项目还将通过制作一个独特类型的示范手势数据库，使其他研究人员受益，该数据库将在公共项目网页上提供。它还将为学生提供独特的教育机会，并根据围绕研究主题开发或改进的新课程为跨学科教育计划做出贡献。

The research goal of this project is to develop new techniques for producing realistic and parameterized humanlike gestures based on motion data acquired from real performances. This work proposes novel computational models and interactive interfaces, and focuses on whole-body demonstrative gestures for interactive training and assistance applications with autonomous virtual humans. Although gesture modeling has made substantial advances in recent years, less attention has been given to parameterized demonstrative gestures which can be modified to refer to arbitrary locations in the environment. This particular class of gestures is critical for a number of applications. Typical examples of such gestures include pointing to and demonstrating how to operate particular devices or objects. To ensure the system's effectiveness, this project includes cognitive studies for guiding the development of the computational gesture model. To ensure the achievement of realistic results, motion capture data obtained from real performers executing gestures for demonstration tasks within real scenarios will be employed. The proposed framework will also account for gestures captured interactively from a low-cost wearable set of motion sensors, enabling the interactive customization of gestures needed for programming interactive virtual human demonstrators for a broad range of applications. This project will significantly advance research on gesture modeling with two key contributions: (1) a novel computational model which integrates blending of realistic full-body gestures from motion capture with motion modification techniques for achieving precise arbitrary placement of the hands at the gesture stroke time, and (2) a new user interface for gesture modeling based on direct demonstrations which will truly enable a seamless human-centered user interface for programming autonomous characters. The approach constitutes a substantial step toward achieving autonomous virtual assistants which can meaningfully and effectively demonstrate tasks and procedures. The interactive interface component of this project has the transformative potential to enable gesture programming to become accessible to the non-specialized user, and therefore to enable virtual humans to become widely employed as a powerful communication medium. This project has the potential to impact the basic and broad research problem of modeling human movement and cognition, which is a central topic in information technology. This project will in addition benefit other researchers by producing a unique type of demonstrative gestures database which will be made available from a public project webpage. It will also provide unique educational opportunities for students and contribute to interdisciplinary educational programs based on new courses being developed or improved around the topics of the research.