CHS: Small: Investigating an Interactive Computational Framework for Nonverbal Interpersonal Skills Training

CHS：小：研究非语言人际技能训练的交互式计算框架

• 批准号: 1421330
• 负责人: Stefan Scherer
• 金额: $ 50万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-01 至 2018-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1421330&HistoricalAwards=false
• 关键词: CHS; Small; Investigating; Interactive; Computational

This project will advance social skill training by developing and evaluating a multimodal computational framework specifically targeted to improve public speaking performance through repeated training interactions with a virtual audience that perceives the speaker and produces meaningful nonverbal feedback. Interpersonal skills such as public speaking are essential assets for a large variety of professions and in everyday life. The ability to communicate in social environments often greatly influences a person's career development, can help build relationships and resolve conflict. Public speaking is not a skill that is innate to everyone, but can be mastered through extensive training. Nonverbal communication is an important aspect of successful public speaking and interpersonal communication, and at the same time difficult to train. This research effort will create the computational foundations to automatically assess interpersonal skill expertise and help people improve their skills using an interactive simulated virtual human framework.There are three fundamental research goals: (1) Developing a probabilistic computational model to learn the temporal and multimodal dependencies and infer a speaker's public speaking performance from acoustic and visual nonverbal behavior; (2) Understanding the design challenges of developing a simulated audience that is interactive, believable, and most importantly providing meaningful and training-relevant feedback to the speaker; and (3) Understanding the impact of the virtual audience on speakers' performance and learning outcomes by performing a comparative study investigating alternative feedback and training approaches. This work builds upon the promising results of a pilot research study and upon a prototype virtual human infrastructure allowing for the seamless integration of automatically modeled interpersonal skill expertise for flexible virtual human interaction and gesture control.Virtual audiences have the great advantage that their appearance and behavioral patterns can be precisely programmed and systematically presented to pace the interaction. The algorithms developed as part of this research to model temporal and multimodal dependencies will have a broad applicability outside the domain of public speaking assessment, including healthcare applications. The interactive virtual human technology may serve as the basis for novel teaching applications in a wide range of areas in the future, due to its extensibility and availability. The programming code and data will be made available to the research community and students.

该项目将通过开发和评估多模式计算框架来推进社交技能培训，该框架专门旨在通过与虚拟听众的反复培训互动来提高公开演讲的表现，虚拟听众能够感知演讲者并产生有意义的非语言反馈。人际交往技能，如公开演讲，对于许多职业和日常生活都是必不可少的资产。在社交环境中沟通的能力往往会极大地影响一个人的职业发展，有助于建立关系和解决冲突。公开演讲不是每个人与生俱来的技能，但可以通过广泛的培训掌握。非语言交际是成功的公共演讲和人际交流的一个重要方面，但同时也很难训练。这项研究将为自动评估人际技能专长并帮助人们使用交互式模拟虚拟人框架来提高他们的技能奠定基础。研究的三个基本目标：(1)开发一个概率计算模型来学习时间和多模式依赖关系，并从听觉和视觉非言语行为中推断演讲者的公开演讲表现；(2)了解建立一个交互的、可信的模拟听众的设计挑战，最重要的是向演讲者提供有意义的和与培训相关的反馈；(3)通过进行一项比较研究，调查替代的反馈和培训方法，了解虚拟听众对演讲者表现和学习结果的影响。这项工作建立在试验性研究的有希望的结果和原型虚拟人类基础设施的基础上，允许自动建模的人际技能专业知识的无缝集成，以实现灵活的虚拟人类交互和手势控制。虚拟观众的巨大优势在于，他们的外表和行为模式可以被精确地编程和系统地呈现，以控制交互的节奏。作为这项研究的一部分，为建模时间和多模式依赖而开发的算法将在公共演讲评估领域之外具有广泛的适用性，包括医疗保健应用。由于其可扩展性和可用性，交互式虚拟人技术可能成为未来在广泛领域的新型教学应用的基础。编程代码和数据将提供给研究社区和学生。