HCC:Small:Computational Studies of Social Nonverbal Communication

点击翻译按钮获取中文摘要

This research will create a new generation of computational tools, called contextual prediction models, for analyzing and modeling social nonverbal communication in human-centered computing. This computational study of nonverbal communication not only encompass the recent advances in machine learning, pattern analysis and computer vision, but goes further by developing and evaluating new algorithms and probabilistic models specifically designed for the domain of social and nonverbal communication. The ability to collect, analyze and ultimately predict human nonverbal cues will provide new insights into human social processes and new human-centric applications that can understand and respond to this natural human communicative channel. This new endeavor will advance through the development of prediction models and their accompanying selection algorithms and feature representations for predicting human nonverbal behavior given a social context (such as the immediately preceding verbal and nonverbal behaviors of a conversational partner). The investigator's previous work has demonstrated the feasibility of using machine learning approaches to model nonverbal communication. Probabilistic sequential models were shown to improve performance of nonverbal behavior recognition during human-robot interactions and make possible the natural animation of virtual humans. This project addresses three fundamental challenges directly: feature representation (optimal mathematical representation of social context), feature selection (subset of social context relevant to prediction of nonverbal behaviors) and probabilistic modeling (efficiently learning the predictive relationship between social context and nonverbal behaviors). This research will evaluate and test the generalization of the computation tools using a large corpus of natural interactions in different settings (human-human, human-robot and human-computer) and domains (e.g., storytelling, interview, and meetings). These prediction models will have broad applicability, including the improvement of nonverbal behavior recognition, the synthesis of natural animations for robots and virtual humans, the training of cultural-specific nonverbal behaviors, and the diagnoses of social disorders (e.g., autism spectrum disorder). The code resulting from this work will be made available to the research community through an open-source Matlab toolbox. The outcome of this research effort will produce state-of-the-art computational models more accessible to researchers who aim to analyze social nonverbal communication and develop natural and productive human-centered computing technologies.