Cognitive based computational modelling of human expression

点击翻译按钮获取中文摘要

The growing success of dialogue systems research makes conversational agents a perfect candidate for becoming a standard in human computer interaction. The naturalness of communicative acts allows for providing a comfortable ground for the users to interact with. There have been many advances on using multiple communication channels in dialogue systems in the way of simulating humaneness in an artificial agent. However, engineering the usability of the system might have contradicting priorities compared with modeling an emotive believable character agent. One issue is to be able to find a good balance of the intensity and frequency of multimodal affective feedback to guide the dialogue flow. The timing and the quality of the feedback can have varying effects on different users. In this work, we propose a framework to incorporate affective input from the user while not sacrificing the perceived interactivity. We use conversational mirroring mechanisms as a way to generate baselines for interaction, that can be used to dynamically guide the user towards the end goal. The proposed research program will study the human behavior while interacting with these assistive technologies using natural interaction methods in order to create an Embodied Conversational Agent (ECA) that enables users to efficiently achieve their goals. The proposed research will develop new theories, models, algorithms, and technologies to make it easier for users to interact more naturally with less cognitive effort. This work will also improve the fundamental understanding of how such systems affect humans. Highly qualified personnel (HQP) will be trained in artificial intelligence (AI) and human computer interaction (HCI), particularly in natural interaction models. Our system uses a fully body realistic 3D avatar with full facial and gesture parameterized animated control. Building on our virtual avatar system that interacts with humans we will use multimodal affective interaction techniques in a complex framework to coordinate the different techniques in a meaningful way. An example of achieving this type of coordination can be seen in previous work that controls gaze, bodily and facial gestures. Our Objectives are: 1. Developing a framework for an affective conversational agent that can recognize and express multimodal emotional cues. 2. Understanding user perception and expectations of using natural interaction methods and bio-feedback mechanisms while interacting with artificial agents. 3. Understanding the effect of the visual and auditory cues on affective human-computer interaction. 4. Apply the observed user parameters to the framework to provide a personalized, user-aware system that can answer varying user needs. 5. With foundational system fork to specific cognitive social models for ECAs including empathy systems and caregiver system.