HCC:Small:Computational Studies of Social Nonverbal Communication

HCC：小：社会非语言交流的计算研究

• 批准号: 0917321
• 负责人: Louis-Philippe Morency
• 金额: $ 49.59万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2014-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0917321&HistoricalAwards=false
• 关键词: HCC; Small; Computational; Studies; Social

This research will create a new generation of computational tools, called contextual prediction models, for analyzing and modeling social nonverbal communication in human-centered computing. This computational study of nonverbal communication not only encompass the recent advances in machine learning, pattern analysis and computer vision, but goes further by developing and evaluating new algorithms and probabilistic models specifically designed for the domain of social and nonverbal communication. The ability to collect, analyze and ultimately predict human nonverbal cues will provide new insights into human social processes and new human-centric applications that can understand and respond to this natural human communicative channel. This new endeavor will advance through the development of prediction models and their accompanying selection algorithms and feature representations for predicting human nonverbal behavior given a social context (such as the immediately preceding verbal and nonverbal behaviors of a conversational partner). The investigator's previous work has demonstrated the feasibility of using machine learning approaches to model nonverbal communication. Probabilistic sequential models were shown to improve performance of nonverbal behavior recognition during human-robot interactions and make possible the natural animation of virtual humans. This project addresses three fundamental challenges directly: feature representation (optimal mathematical representation of social context), feature selection (subset of social context relevant to prediction of nonverbal behaviors) and probabilistic modeling (efficiently learning the predictive relationship between social context and nonverbal behaviors). This research will evaluate and test the generalization of the computation tools using a large corpus of natural interactions in different settings (human-human, human-robot and human-computer) and domains (e.g., storytelling, interview, and meetings). These prediction models will have broad applicability, including the improvement of nonverbal behavior recognition, the synthesis of natural animations for robots and virtual humans, the training of cultural-specific nonverbal behaviors, and the diagnoses of social disorders (e.g., autism spectrum disorder). The code resulting from this work will be made available to the research community through an open-source Matlab toolbox. The outcome of this research effort will produce state-of-the-art computational models more accessible to researchers who aim to analyze social nonverbal communication and develop natural and productive human-centered computing technologies.

这项研究将创建新一代的计算工具，称为上下文预测模型，用于分析和建模以人为中心的计算中的社会非语言交流。这项对非语言交流的计算研究不仅涵盖了机器学习、模式分析和计算机视觉领域的最新进展，而且通过开发和评估专门为社交和非语言交流领域设计的新算法和概率模型而更进一步。收集、分析并最终预测人类非语言线索的能力将为人类社会过程和以人为中心的新应用提供新的见解，这些应用可以理解并响应这种自然的人类沟通渠道。这一新的奋进将通过开发预测模型及其伴随的选择算法和特征表示来推进，以预测给定社会背景下的人类非语言行为（例如会话伙伴的前一个语言和非语言行为）。研究人员以前的工作已经证明了使用机器学习方法来模拟非语言交流的可行性。概率序列模型被证明可以提高人机交互过程中非语言行为识别的性能，并使虚拟人的自然动画成为可能。该项目直接解决了三个基本挑战：特征表示（社会背景的最佳数学表示），特征选择（与非语言行为预测相关的社会背景子集）和概率建模（有效地学习社会背景和非语言行为之间的预测关系）。这项研究将使用不同环境（人-人，人-机器人和人机）和领域（例如，讲故事、采访和会议）。这些预测模型将具有广泛的适用性，包括非语言行为识别的改进，机器人和虚拟人的自然动画的合成，特定文化非语言行为的训练，以及社会障碍的诊断（例如，自闭症谱系障碍）。这项工作产生的代码将通过开源Matlab工具箱提供给研究界。这项研究工作的成果将产生最先进的计算模型，更容易为那些旨在分析社会非语言交流和开发自然和富有成效的以人为本的计算技术的研究人员所用。