HCC: Small: Body Language Animation for Virtual Worlds and Computer-Mediated Communication

HCC：小：虚拟世界和计算机介导的交流的肢体语言动画

• 批准号: 1017938
• 负责人: Vladlen Koltun
• 金额: $ 49.52万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1017938&HistoricalAwards=false
• 关键词: HCC; Small; Body; Language; Animation

This project will enable full-body nonverbal communication in virtual worlds, applying state-of-the-art machine learning and computer animation techniques to the synthesis of nonverbal communication, advancing and refining these techniques in the process. Virtual worlds are an emerging computer-mediated communication medium that situates geographically distributed participants in a shared communication space and enables embodied interaction with others in a simulated environment. Participants are represented as animated virtual humans that can convey both speech and body language. Yet no viable technology exists that can animate the virtual human's body during a live conversation without resorting to esoteric hardware or brittle algorithmic techniques.This research will develop an approach that can convey body language through virtual humans in real time, using a natural control interface: the speech and motion of the participants. The proposal is based on, and sometimes advances, the state of the art in machine learning, computer animation, and the relevant aspects of linguistics and cognitive psychology. Body language animation based purely on visual tracking of the participant's motion is prone to significant defects due to tracking noise and failure. Thus this approach analyzes the speech of the participant together with the motion. This principled integration of live speech and motion input constitutes a fundamentally new approach to the control of nonverbal expression of human self-representations in virtual worlds.The ability to convey rich nonverbal communication in virtual worlds will advance the capabilities of computer-mediated communication as a whole. This will provide basic infrastructure for distributed collaboration in science and engineering. Powerful forms of situated learning and social-scientific inquiry in education will be enabled, with positive impact on the self-efficacy of students who traditionally underperform in science curricula. Social science will be enriched with a new medium for the study of human interaction.

该项目将在虚拟世界中实现全身非语言交流，将最先进的机器学习和计算机动画技术应用于非语言交流的合成，并在此过程中推进和完善这些技术。虚拟世界是一种新兴的以计算机为媒介的通信媒介，它将地理上分布的参与者置于共享的通信空间中，并使其能够在模拟环境中与他人进行具体的交互。参与者被表示为可以传达语音和肢体语言的动画虚拟人。然而，目前还没有一种可行的技术可以在不借助深奥的硬件或脆弱的算法技术的情况下，在现场对话中使虚拟人的身体动画化。本研究将开发一种方法，可以通过虚拟人在真实的时间内传达肢体语言，使用自然的控制界面：参与者的语音和动作。该提案基于机器学习、计算机动画以及语言学和认知心理学的相关方面的最新技术，有时甚至是先进技术。纯粹基于对参与者运动的视觉跟踪的肢体语言动画由于跟踪噪声和失败而容易出现显著缺陷。因此，这种方法分析了参与者的语音和运动。这种实时语音和动作输入的原则性整合构成了一种全新的方法来控制虚拟世界中人类自我表征的非语言表达，在虚拟世界中传达丰富的非语言交流的能力将推动计算机媒介交流的整体能力。这将为科学和工程领域的分布式协作提供基本的基础设施。将在教育中启用强有力的情境学习和社会科学探究形式，对传统上在科学课程中表现不佳的学生的自我效能产生积极影响。社会科学将因研究人类互动的新媒介而丰富。