user adaPtive Artificial iNtelligence fOR humAn coMputer interAction

PANORAMA的核心概念是“人机交互背景下的用户自适应人工智能”。首先，我们将研究作为会话代理的人工智能的用户适应性。当人们与他人交谈时，他们会根据对方的行为不断改变自己的语言和非语言交流行为。因此，用户自适应是改善人机交互的关键问题。沟通风格也因文化的不同而不同，在系统本地化中需要使agent的行为适应目标文化。PANORAMA将用机器学习解决这些问题。然而，这种方法的一个瓶颈是注释用户的非语言行为来创建训练数据非常耗时。我们将利用可解释的人工智能（XAI）技术来解决这个问题，通过该技术，系统预测的标签将根据与作为注释者的用户的交互进行调整。因此，用户自适应人工智能能够支持用户创建多模态语料库，并改善人机交互。此外，PANORAMA通过心理学理论考虑了用户适应性，其中用户动机将在一个相关用例中进行调查（针对体育活动的个性化动机指导）。因此，PANORAMA通过关注用户适应性的概念，为基于机器学习的特定文化对话代理设想了一种新的研究方法。PANORAMA旨在完成以下五个研究目标。(1)提出了一种基于XAI技术的用户自适应多模态标注工具；(2)利用该工具在法国、德国和日本三个国家收集标注的多模态语料库；(3)提出了开发具有多级自适应功能的会话代理的模型和方法，其中代理的非语言信号以及对话内容被适应于用户；(4)提供多任务学习和迁移学习技术，使用(2)中获得的多文化语料库学习模型，并使会话代理适应每种文化；(5)提出基于心理学理论和评估研究的自适应人工智能系统的设计基础。

The key concept of PANORAMA is "user adaptive AI in the context of human-computer interaction". First, we will conduct research on user adaptivity of Artificial Intelligence embodied as a conversational agent. When people talk to other people, they change their verbal and nonverbal communication behaviors continuously according to those of the partner. Therefore, user adaptivity is an essential issue in improving human-agent interaction. Communication style is also different depending on the culture, and adapting the agent behaviors to a target culture is required in system localization. PANORAMA will tackle these problems with Machine Learning. However, a bottleneck of this approach is that annotating users’ nonverbal behaviors to create training data is time consuming. We will solve this problem by exploiting Explainable Artificial Intelligence (XAI) technique, through which labels predicted by the system are adapted based on the interaction with the user as an annotator. Thus, user adaptive AI enables to support users in creating multimodal corpus as well as improve human-agent interaction. Moreover, user adaptivity is considered in PANORAMA via psychological theories, in which user motivation will be investigated in one relevant use case (personalised motivational coaching for physical activity). Therefore, PANORAMA envisions a new research methodology for Machine-Learning-based culture-specific conversational agents by focusing on the concept of user adaptivity.PANORAMA aims to accomplish the following five research goals. (1) propose a user adaptive multimodal annotation tool based on XAI techniques, (2) exploit this tool to collect annotated multimodal corpora in three countries (France, Germany, and Japan), (3) propose models and methods for developing conversational agents with multi-level adaptation functionality, where nonverbal signals of the agent as well as the content of the dialogue are adapted to the user, (4) provide multitask learning and transfer learning techniques to learn models using the multi-cultural corpus obtained in (2) and adapt the conversational agent to each culture, and (5) propose the design basis of adaptive AI systems grounded in psychological theories and evaluation studies.