CHS: Small: Investigating an Interactive Computational Framework for Nonverbal Interpersonal Skills Training

该项目将通过开发和评估一个多模态计算框架来推进社交技能培训，该框架专门针对通过与虚拟观众的重复培训互动来提高公众演讲表现，该虚拟观众感知演讲者并产生有意义的非语言反馈。 人际交往技能，如公开演讲，是各种职业和日常生活中的重要资产。 在社会环境中沟通的能力往往会极大地影响一个人的职业发展，可以帮助建立关系和解决冲突。 演讲不是每个人与生俱来的技能，而是可以通过广泛的训练来掌握的。 非言语交际是成功的演讲和人际交往的重要方面，同时也是很难训练的。 本研究的主要目的是建立一个基于概率的计算模型，以学习演讲者的时间依赖和多模态依赖，并从声音和视觉非语言行为中推断演讲者的演讲表现;（2）了解设计的挑战，开发一个模拟的观众，是互动的，可信的，最重要的是提供有意义的和培训相关的反馈给扬声器;和（3）了解的影响，虚拟观众对扬声器的性能和学习成果进行比较研究调查替代反馈和培训方法。 这项工作建立在一个试点研究的可喜成果和原型虚拟人的基础设施，允许无缝集成的自动建模的人际交往技能的专业知识，灵活的虚拟人的互动和手势controlling.Virtual观众有很大的优势，他们的外观和行为模式可以精确编程和系统地提出了节奏的互动。 作为本研究的一部分，开发的算法对时间和多模态依赖关系进行建模，将在公共演讲评估领域之外具有广泛的适用性，包括医疗保健应用。 交互式虚拟人技术由于其可扩展性和可用性，可以作为未来广泛领域的新型教学应用的基础。 编程代码和数据将提供给研究界和学生。

This project will advance social skill training by developing and evaluating a multimodal computational framework specifically targeted to improve public speaking performance through repeated training interactions with a virtual audience that perceives the speaker and produces meaningful nonverbal feedback. Interpersonal skills such as public speaking are essential assets for a large variety of professions and in everyday life. The ability to communicate in social environments often greatly influences a person's career development, can help build relationships and resolve conflict. Public speaking is not a skill that is innate to everyone, but can be mastered through extensive training. Nonverbal communication is an important aspect of successful public speaking and interpersonal communication, and at the same time difficult to train. This research effort will create the computational foundations to automatically assess interpersonal skill expertise and help people improve their skills using an interactive simulated virtual human framework.There are three fundamental research goals: (1) Developing a probabilistic computational model to learn the temporal and multimodal dependencies and infer a speaker's public speaking performance from acoustic and visual nonverbal behavior; (2) Understanding the design challenges of developing a simulated audience that is interactive, believable, and most importantly providing meaningful and training-relevant feedback to the speaker; and (3) Understanding the impact of the virtual audience on speakers' performance and learning outcomes by performing a comparative study investigating alternative feedback and training approaches. This work builds upon the promising results of a pilot research study and upon a prototype virtual human infrastructure allowing for the seamless integration of automatically modeled interpersonal skill expertise for flexible virtual human interaction and gesture control.Virtual audiences have the great advantage that their appearance and behavioral patterns can be precisely programmed and systematically presented to pace the interaction. The algorithms developed as part of this research to model temporal and multimodal dependencies will have a broad applicability outside the domain of public speaking assessment, including healthcare applications. The interactive virtual human technology may serve as the basis for novel teaching applications in a wide range of areas in the future, due to its extensibility and availability. The programming code and data will be made available to the research community and students.