HCC: Small: Collaborative Research: Gestural and Linguistic Expressivity and Entrainment in Dialogue

点击翻译按钮获取中文摘要

This research will advance the state of the art in conversational character systems (Intelligent Virtual Agents) in two ways. First, it seeks to better understand the interplay of gesture and language in both generating the perception of personality and allowing participants to adapt to the ongoing conversational context. Second, it will use this understanding to build novel computational models for gesture and language generation that provide fine grained control over the perception of personality and support agent adaptation in response to the conversational context. The theoretical basis for the personality-based modeling in this project is the well-established "Big Five" model of personality, which consists of five orthogonal dimensions of individual variation.The work on adaptation will be couched in the collaborative theory of language use and communication accommodation theory, which predict that communicative behavior varies based on partner specificity. Initial work will form a motion capture, video and audio corpus of three kinds of exchanges. This will be used to both study gestural entrainment during human interactions, determining if audio-based findings extend to the gestural domain, and to enhance scientific understanding of the relationship between gesture and personality. This will inform the modeling work which will build a joint model for personality-based language and gesture production. The model will extend a pilot study on gesture generation for extraversion to three Big Five traits and integrate it with personality-based language generation. An experimental stage will validate these models and study the interactions of movement and language. The research will also study the role of adaptation. Questions to be answered include: (1) whether people gesturally entrain with computers, or indeed produce any gestures while communicating with a computer, (2) whether computers? gestural entrainment promotes similar levels of affiliation as observed with vocal entrainment, and (3) whether changing gestural entrainment over the course of an interaction is more powerful than aligning gestures from the outset of an interaction. Character systems are becoming increasingly important for a range of applications, from virtual worlds to tutoring systems. There is growing evidence that the way personality is presented through these characters, and how well they mimic expected human behavior like entrainment, has a direct impact on the effectiveness of the applications in which they are used. For example, it will have a direct impact on student learning. As these applications become more ubiquitous in society, particularly among children, it is important to be able to harness their full benefit, and indeed, avoid unintended negative consequences. This involves both advances in computational models that allow an agent to reflect a given personality and adapt to a human user, and also a deeper understanding of the role of personality and adaptation in effective human-agent interactions.