HCC:Small:Computational Studies of Social Nonverbal Communication

这项研究将创造新一代计算工具，称为语境预测模型，用于在以人为中心的计算中分析和建模社交非语言交流。这项非语言交流的计算研究不仅涵盖了机器学习、模式分析和计算机视觉的最新进展，而且通过开发和评估专门为社会和非语言交流领域设计的新算法和概率模型而更进一步。收集、分析和最终预测人类非语言线索的能力将为人类社交过程提供新的见解，并提供新的以人类为中心的应用程序，这些应用程序可以理解和响应这一自然的人类交流渠道。这一新的努力将通过发展预测模型及其伴随的选择算法和特征表示来预测给定的社会背景下的人类非语言行为(例如对话伙伴的紧随其前的语言和非语言行为)。这位研究人员之前的工作已经证明了使用机器学习方法来模拟非语言交流的可行性。概率序列模型提高了人机交互过程中非语言行为识别的性能，使虚拟人的自然动画成为可能。这个项目直接解决了三个基本挑战：特征表示(社会语境的最佳数学表示)、特征选择(与非言语行为预测相关的社会语境子集)和概率建模(有效地学习社会语境和非言语行为之间的预测关系)。这项研究将使用不同环境(人-人、人-机器人和人-计算机)和领域(例如，讲故事、采访和会议)的自然交互的大型语料库来评估和测试计算工具的通用性。这些预测模型将具有广泛的适用性，包括改进非语言行为识别，为机器人和虚拟人合成自然动画，训练特定文化的非语言行为，以及诊断社交障碍(如自闭症谱系障碍)。这项工作产生的代码将通过开放源码的MatLab工具箱提供给研究界。这项研究的成果将产生最先进的计算模型，更容易为研究人员所使用，他们的目标是分析社会非语言交流，开发自然和富有成效的以人为中心的计算技术。

This research will create a new generation of computational tools, called contextual prediction models, for analyzing and modeling social nonverbal communication in human-centered computing. This computational study of nonverbal communication not only encompass the recent advances in machine learning, pattern analysis and computer vision, but goes further by developing and evaluating new algorithms and probabilistic models specifically designed for the domain of social and nonverbal communication. The ability to collect, analyze and ultimately predict human nonverbal cues will provide new insights into human social processes and new human-centric applications that can understand and respond to this natural human communicative channel. This new endeavor will advance through the development of prediction models and their accompanying selection algorithms and feature representations for predicting human nonverbal behavior given a social context (such as the immediately preceding verbal and nonverbal behaviors of a conversational partner). The investigator's previous work has demonstrated the feasibility of using machine learning approaches to model nonverbal communication. Probabilistic sequential models were shown to improve performance of nonverbal behavior recognition during human-robot interactions and make possible the natural animation of virtual humans. This project addresses three fundamental challenges directly: feature representation (optimal mathematical representation of social context), feature selection (subset of social context relevant to prediction of nonverbal behaviors) and probabilistic modeling (efficiently learning the predictive relationship between social context and nonverbal behaviors). This research will evaluate and test the generalization of the computation tools using a large corpus of natural interactions in different settings (human-human, human-robot and human-computer) and domains (e.g., storytelling, interview, and meetings). These prediction models will have broad applicability, including the improvement of nonverbal behavior recognition, the synthesis of natural animations for robots and virtual humans, the training of cultural-specific nonverbal behaviors, and the diagnoses of social disorders (e.g., autism spectrum disorder). The code resulting from this work will be made available to the research community through an open-source Matlab toolbox. The outcome of this research effort will produce state-of-the-art computational models more accessible to researchers who aim to analyze social nonverbal communication and develop natural and productive human-centered computing technologies.