HCC: Small: Body Language Animation for Virtual Worlds and Computer-Mediated Communication

该项目将在虚拟世界中实现全身非语言交流，将最先进的机器学习和计算机动画技术应用于非语言交流的合成，在此过程中改进和完善这些技术。虚拟世界是一种新兴的计算机中介通信媒介，它将地理上分布的参与者置于共享的通信空间中，并使其能够在模拟环境中与他人进行具体化的交互。参与者被表示为既能传达语音又能传达肢体语言的动画虚拟人。然而，没有一种可行的技术可以在不诉诸深奥的硬件或脆弱的算法技术的情况下，在实时对话中激活虚拟人的身体。这项研究将开发一种方法，可以通过虚拟人实时传达肢体语言，使用一个自然的控制界面：参与者的语音和动作。这项建议是基于机器学习、计算机动画以及语言学和认知心理学相关方面的最新水平，有时甚至是进步的。单纯基于对参与者运动的视觉跟踪的肢体语言动画容易由于跟踪噪声和失败而存在明显的缺陷。因此，这种方法将参与者的发言与动作一起分析。这种现场语音和动作输入的原则性集成构成了一种全新的方法来控制虚拟世界中人类自我表征的非语言表达。在虚拟世界中传达丰富的非语言交流的能力将从整体上促进计算机中介交流的能力。这将为科学和工程领域的分布式协作提供基本的基础设施。将在教育中启用强大的情境学习和社会科学探究形式，对传统上在科学课程中表现不佳的学生的自我效能产生积极影响。社会科学将被一种研究人类互动的新媒介所丰富。

This project will enable full-body nonverbal communication in virtual worlds, applying state-of-the-art machine learning and computer animation techniques to the synthesis of nonverbal communication, advancing and refining these techniques in the process. Virtual worlds are an emerging computer-mediated communication medium that situates geographically distributed participants in a shared communication space and enables embodied interaction with others in a simulated environment. Participants are represented as animated virtual humans that can convey both speech and body language. Yet no viable technology exists that can animate the virtual human's body during a live conversation without resorting to esoteric hardware or brittle algorithmic techniques.This research will develop an approach that can convey body language through virtual humans in real time, using a natural control interface: the speech and motion of the participants. The proposal is based on, and sometimes advances, the state of the art in machine learning, computer animation, and the relevant aspects of linguistics and cognitive psychology. Body language animation based purely on visual tracking of the participant's motion is prone to significant defects due to tracking noise and failure. Thus this approach analyzes the speech of the participant together with the motion. This principled integration of live speech and motion input constitutes a fundamentally new approach to the control of nonverbal expression of human self-representations in virtual worlds.The ability to convey rich nonverbal communication in virtual worlds will advance the capabilities of computer-mediated communication as a whole. This will provide basic infrastructure for distributed collaboration in science and engineering. Powerful forms of situated learning and social-scientific inquiry in education will be enabled, with positive impact on the self-efficacy of students who traditionally underperform in science curricula. Social science will be enriched with a new medium for the study of human interaction.