HCC:Small:Computational Studies of Social Nonverbal Communication

这项研究将创造新一代的计算工具，称为上下文预测模型，用于分析和模拟以人为中心的计算中的社会非语言交流。非言语交际的计算研究不仅包括机器学习、模式分析和计算机视觉方面的最新进展，而且还通过开发和评估专门为社交和非言语交际领域设计的新算法和概率模型而进一步发展。收集、分析和最终预测人类非语言线索的能力将为人类社会过程和新的以人为中心的应用提供新的见解，这些应用可以理解和响应这种自然的人类交流渠道。这一新的努力将通过预测模型及其伴随的选择算法和特征表征的发展来推进，以预测给定社会背景下的人类非语言行为（例如对话伙伴的直接言语和非言语行为）。研究者之前的工作已经证明了使用机器学习方法模拟非语言交流的可行性。结果表明，概率序列模型可以提高人机交互过程中非语言行为识别的性能，并使虚拟人的自然动画成为可能。该项目直接解决了三个基本挑战：特征表示（社会情境的最佳数学表示）、特征选择（与非语言行为预测相关的社会情境子集）和概率建模（有效学习社会情境和非语言行为之间的预测关系）。本研究将使用不同设置（人-人，人-机器人和人机）和领域（例如，讲故事，采访和会议）中的大量自然交互语料库来评估和测试计算工具的泛化。这些预测模型将具有广泛的适用性，包括改进非语言行为识别，合成机器人和虚拟人的自然动画，训练特定文化的非语言行为，以及诊断社会障碍（如自闭症谱系障碍）。这项工作产生的代码将通过开源Matlab工具箱提供给研究社区。这项研究工作的结果将产生最先进的计算模型，使旨在分析社会非语言交流和开发自然和富有成效的以人为中心的计算技术的研究人员更容易使用。

This research will create a new generation of computational tools, called contextual prediction models, for analyzing and modeling social nonverbal communication in human-centered computing. This computational study of nonverbal communication not only encompass the recent advances in machine learning, pattern analysis and computer vision, but goes further by developing and evaluating new algorithms and probabilistic models specifically designed for the domain of social and nonverbal communication. The ability to collect, analyze and ultimately predict human nonverbal cues will provide new insights into human social processes and new human-centric applications that can understand and respond to this natural human communicative channel. This new endeavor will advance through the development of prediction models and their accompanying selection algorithms and feature representations for predicting human nonverbal behavior given a social context (such as the immediately preceding verbal and nonverbal behaviors of a conversational partner). The investigator's previous work has demonstrated the feasibility of using machine learning approaches to model nonverbal communication. Probabilistic sequential models were shown to improve performance of nonverbal behavior recognition during human-robot interactions and make possible the natural animation of virtual humans. This project addresses three fundamental challenges directly: feature representation (optimal mathematical representation of social context), feature selection (subset of social context relevant to prediction of nonverbal behaviors) and probabilistic modeling (efficiently learning the predictive relationship between social context and nonverbal behaviors). This research will evaluate and test the generalization of the computation tools using a large corpus of natural interactions in different settings (human-human, human-robot and human-computer) and domains (e.g., storytelling, interview, and meetings). These prediction models will have broad applicability, including the improvement of nonverbal behavior recognition, the synthesis of natural animations for robots and virtual humans, the training of cultural-specific nonverbal behaviors, and the diagnoses of social disorders (e.g., autism spectrum disorder). The code resulting from this work will be made available to the research community through an open-source Matlab toolbox. The outcome of this research effort will produce state-of-the-art computational models more accessible to researchers who aim to analyze social nonverbal communication and develop natural and productive human-centered computing technologies.