Cognitive based computational modelling of human expression

对话系统研究的日益成功使对话代理成为人机交互标准的完美候选。交流行为的自然性允许为用户提供一个舒适的互动环境。在对话系统中使用多种沟通渠道，在人工智能体中模拟人类的方式，已经取得了许多进展。然而，设计系统的可用性可能与建模一个情感可信的角色代理有矛盾的优先级。一个问题是如何在多模态情感反馈的强度和频率之间找到一个很好的平衡，从而引导对话流程。反馈的时间和质量会对不同的用户产生不同的影响。在这项工作中，我们提出了一个框架，在不牺牲感知交互性的情况下，结合用户的情感输入。我们使用会话镜像机制作为生成交互基线的一种方式，它可以用来动态地引导用户走向最终目标。拟议的研究计划将研究人类在使用自然交互方法与这些辅助技术交互时的行为，以创建一个具体化的会话代理（ECA），使用户能够有效地实现他们的目标。拟议的研究将开发新的理论、模型、算法和技术，使用户更容易以更少的认知努力进行更自然的交互。这项工作还将提高对这些系统如何影响人类的基本理解。高素质人才（HQP）将接受人工智能（AI）和人机交互（HCI）方面的培训，特别是在自然交互模型方面。我们的系统使用了一个全身逼真的3D化身，具有完整的面部和手势参数化动画控制。在与人类交互的虚拟化身系统的基础上，我们将在一个复杂的框架中使用多模态情感交互技术，以一种有意义的方式协调不同的技术。实现这种类型的协调的一个例子可以在之前的研究中看到，控制凝视，身体和面部手势。我们的目标是：1；开发一个能够识别和表达多模态情感线索的情感会话代理框架。2. 了解用户在与人工智能体交互时使用自然交互方法和生物反馈机制的感知和期望。3. 理解视觉和听觉线索对情感人机交互的影响。4. 将观察到的用户参数应用到框架中，以提供可满足不同用户需求的个性化、用户感知的系统。5. 从基础系统到具体的认知社会模型，包括移情系统和照顾者系统。

The growing success of dialogue systems research makes conversational agents a perfect candidate for becoming a standard in human computer interaction. The naturalness of communicative acts allows for providing a comfortable ground for the users to interact with. There have been many advances on using multiple communication channels in dialogue systems in the way of simulating humaneness in an artificial agent. However, engineering the usability of the system might have contradicting priorities compared with modeling an emotive believable character agent. One issue is to be able to find a good balance of the intensity and frequency of multimodal affective feedback to guide the dialogue flow. The timing and the quality of the feedback can have varying effects on different users. In this work, we propose a framework to incorporate affective input from the user while not sacrificing the perceived interactivity. We use conversational mirroring mechanisms as a way to generate baselines for interaction, that can be used to dynamically guide the user towards the end goal. The proposed research program will study the human behavior while interacting with these assistive technologies using natural interaction methods in order to create an Embodied Conversational Agent (ECA) that enables users to efficiently achieve their goals. The proposed research will develop new theories, models, algorithms, and technologies to make it easier for users to interact more naturally with less cognitive effort. This work will also improve the fundamental understanding of how such systems affect humans. Highly qualified personnel (HQP) will be trained in artificial intelligence (AI) and human computer interaction (HCI), particularly in natural interaction models. Our system uses a fully body realistic 3D avatar with full facial and gesture parameterized animated control. Building on our virtual avatar system that interacts with humans we will use multimodal affective interaction techniques in a complex framework to coordinate the different techniques in a meaningful way. An example of achieving this type of coordination can be seen in previous work that controls gaze, bodily and facial gestures. Our Objectives are: 1. Developing a framework for an affective conversational agent that can recognize and express multimodal emotional cues. 2. Understanding user perception and expectations of using natural interaction methods and bio-feedback mechanisms while interacting with artificial agents. 3. Understanding the effect of the visual and auditory cues on affective human-computer interaction. 4. Apply the observed user parameters to the framework to provide a personalized, user-aware system that can answer varying user needs. 5. With foundational system fork to specific cognitive social models for ECAs including empathy systems and caregiver system.