Cognitive based computational modelling of human expression

对话系统研究的日益成功使得会话代理成为人机交互标准的完美候选者。交际行为的自然性为用户提供了一个舒适的互动环境。在对话系统中使用多个通信通道来模拟人工智能体中的人性已经取得了许多进展。然而，工程系统的可用性可能有矛盾的优先级相比，建模一个情感可信的性格代理。一个问题是要能够找到一个很好的平衡的强度和频率的多模态情感反馈，以指导对话流。反馈的时间和质量可能对不同的用户产生不同的影响。在这项工作中，我们提出了一个框架，将用户的情感输入，同时不牺牲感知的交互性。我们使用会话镜像机制作为生成交互基线的一种方式，可以用来动态地引导用户实现最终目标。拟议的研究计划将研究人类的行为，同时使用自然的交互方法与这些辅助技术进行交互，以创建一个嵌入式会话代理（ECA），使用户能够有效地实现他们的目标。拟议的研究将开发新的理论、模型、算法和技术，使用户更容易以更少的认知努力更自然地进行交互。这项工作还将提高对这些系统如何影响人类的基本理解。高素质人员（HQP）将接受人工智能（AI）和人机交互（HCI）的培训，特别是自然交互模型。我们的系统使用了一个完整的身体逼真的3D头像与完整的面部和姿态参数化动画控制。在我们与人类互动的虚拟化身系统的基础上，我们将在一个复杂的框架中使用多模态情感交互技术，以有意义的方式协调不同的技术。实现这种协调的一个例子可以在以前的工作中看到，控制凝视，身体和面部姿态。我们的目标是：*1。开发一个情感会话代理的框架，可以识别和表达多模态情感线索。2.了解用户对使用自然交互方法和生物反馈机制的感知和期望，同时与人工代理进行交互。** 3.理解视觉和听觉线索对情感人机交互的影响。4.将观察到的用户参数应用于框架，以提供个性化的、用户感知的系统，该系统可以满足不同的用户需求。5.从基础系统出发，探讨了情感关怀的具体认知社会模型，包括移情系统和照顾者系统。

The growing success of dialogue systems research makes conversational agents a perfect candidate for becoming a standard in human computer interaction. The naturalness of communicative acts allows for providing a comfortable ground for the users to interact with. There have been many advances on using multiple communication channels in dialogue systems in the way of simulating humaneness in an artificial agent. However, engineering the usability of the system might have contradicting priorities compared with modeling an emotive believable character agent. One issue is to be able to find a good balance of the intensity and frequency of multimodal affective feedback to guide the dialogue flow. The timing and the quality of the feedback can have varying effects on different users. In this work, we propose a framework to incorporate affective input from the user while not sacrificing the perceived interactivity. We use conversational mirroring mechanisms as a way to generate baselines for interaction, that can be used to dynamically guide the user towards the end goal. The proposed research program will study the human behavior while interacting with these assistive technologies using natural interaction methods in order to create an Embodied Conversational Agent (ECA) that enables users to efficiently achieve their goals. The proposed research will develop new theories, models, algorithms, and technologies to make it easier for users to interact more naturally with less cognitive effort. This work will also improve the fundamental understanding of how such systems affect humans. Highly qualified personnel (HQP) will be trained in artificial intelligence (AI) and human computer interaction (HCI), particularly in natural interaction models. Our system uses a fully body realistic 3D avatar with full facial and gesture parameterized animated control. Building on our virtual avatar system that interacts with humans we will use multimodal affective interaction techniques in a complex framework to coordinate the different techniques in a meaningful way. An example of achieving this type of coordination can be seen in previous work that controls gaze, bodily and facial gestures.******Our Objectives are:******1. Developing a framework for an affective conversational agent that can recognize and express multimodal emotional cues.******2. Understanding user perception and expectations of using natural interaction methods and bio-feedback mechanisms while interacting with artificial agents.******3. Understanding the effect of the visual and auditory cues on affective human-computer interaction.******4. Apply the observed user parameters to the framework to provide a personalized, user-aware system that can answer varying user needs.******5. With foundational system fork to specific cognitive social models for ECAs including empathy systems and caregiver system.