Cognitive based computational modelling of human expression

对话系统研究的日益成功使对话代理成为人机交互的标准。交流行为的自然性允许为用户提供一个舒适的互动场所。在人工智能体中模拟人性化的方式在对话系统中使用多个通信通道已经取得了许多进展。然而，与建立一个情绪化、可信的角色代理相比，设计系统的可用性可能具有相互矛盾的优先级。一个问题是能够在多模式情感反馈的强度和频率之间找到一个很好的平衡，以指导对话流。反馈的时间和质量可能会对不同的用户产生不同的影响。在这项工作中，我们提出了一个框架，在不牺牲感知的互动性的同时，纳入了来自用户的情感输入。我们使用对话镜像机制作为生成交互基线的一种方式，可以用来动态地引导用户朝着最终目标前进。拟议的研究计划将研究人类在使用自然交互方法与这些辅助技术交互时的行为，以创建一个能够使用户高效地实现他们的目标的具体化对话代理(ECA)。这项拟议的研究将开发新的理论、模型、算法和技术，使用户以更少的认知努力更容易地进行更自然的交互。这项工作还将提高对这类系统如何影响人类的基本理解。高素质人员(HQP)将接受人工智能(AI)和人机交互(HCI)方面的培训，特别是自然交互模型方面的培训。我们的系统使用了一个全身逼真的3D化身，带有完整的面部和手势参数动画控制。在我们与人类互动的虚拟化身系统的基础上，我们将在一个复杂的框架中使用多通道情感交互技术，以有意义的方式协调不同的技术。实现这种协调的一个例子可以在以前控制凝视、身体和面部手势的工作中看到。 我们的目标是： 1.开发一个能够识别和表达多通道情绪线索的情感型会话代理框架。 2.了解用户在与人工智能体交互时对使用自然交互方法和生物反馈机制的感知和期望。 3.了解视觉和听觉线索对情感性人机交互的影响。 4.将观察到的用户参数应用于框架，以提供能够响应不同用户需求的个性化的、用户感知的系统。 5.在基础系统的基础上，构建了包括移情系统和照顾者系统在内的ECAs具体认知社会模型。

The growing success of dialogue systems research makes conversational agents a perfect candidate for becoming a standard in human computer interaction. The naturalness of communicative acts allows for providing a comfortable ground for the users to interact with. There have been many advances on using multiple communication channels in dialogue systems in the way of simulating humaneness in an artificial agent. However, engineering the usability of the system might have contradicting priorities compared with modeling an emotive believable character agent. One issue is to be able to find a good balance of the intensity and frequency of multimodal affective feedback to guide the dialogue flow. The timing and the quality of the feedback can have varying effects on different users. In this work, we propose a framework to incorporate affective input from the user while not sacrificing the perceived interactivity. We use conversational mirroring mechanisms as a way to generate baselines for interaction, that can be used to dynamically guide the user towards the end goal. The proposed research program will study the human behavior while interacting with these assistive technologies using natural interaction methods in order to create an Embodied Conversational Agent (ECA) that enables users to efficiently achieve their goals. The proposed research will develop new theories, models, algorithms, and technologies to make it easier for users to interact more naturally with less cognitive effort. This work will also improve the fundamental understanding of how such systems affect humans. Highly qualified personnel (HQP) will be trained in artificial intelligence (AI) and human computer interaction (HCI), particularly in natural interaction models. Our system uses a fully body realistic 3D avatar with full facial and gesture parameterized animated control. Building on our virtual avatar system that interacts with humans we will use multimodal affective interaction techniques in a complex framework to coordinate the different techniques in a meaningful way. An example of achieving this type of coordination can be seen in previous work that controls gaze, bodily and facial gestures. Our Objectives are: 1. Developing a framework for an affective conversational agent that can recognize and express multimodal emotional cues. 2. Understanding user perception and expectations of using natural interaction methods and bio-feedback mechanisms while interacting with artificial agents. 3. Understanding the effect of the visual and auditory cues on affective human-computer interaction. 4. Apply the observed user parameters to the framework to provide a personalized, user-aware system that can answer varying user needs. 5. With foundational system fork to specific cognitive social models for ECAs including empathy systems and caregiver system.