user adaPtive Artificial iNtelligence fOR humAn coMputer interAction

PANORAMA的核心概念是“人机交互背景下的用户自适应AI”。首先，我们将进行人工智能的用户自适应性的研究体现为一个会话代理。当人们与他人交谈时，他们会根据对方的行为不断改变自己的语言和非语言交际行为。因此，用户自适应性是改善人机交互的一个重要问题。沟通风格也因文化的不同而不同，在系统本地化中需要使代理行为适应目标文化。PANORAMA将通过机器学习解决这些问题。然而，这种方法的一个瓶颈是注释用户的非语言行为来创建训练数据是耗时的。我们将通过利用可解释人工智能（XAI）技术来解决这个问题，通过该技术，系统预测的标签将根据与用户的交互进行调整。因此，用户自适应AI能够支持用户创建多模态语料库，并改善人机交互。此外，PANORAMA通过心理学理论考虑了用户适应性，其中将在一个相关用例中研究用户动机（针对身体活动的个性化激励辅导）。因此，PANORAMA通过关注用户自适应性的概念，设想了一种新的基于机器学习的文化特定会话代理的研究方法。PANORAMA旨在实现以下五个研究目标。(1)提出了一个基于XAI技术的用户自适应多模态标注工具，（2）利用该工具收集了三个国家的多模态标注语料（法国、德国和日本），（3）提出用于开发具有多级适应功能的会话代理的模型和方法，其中代理的非语言信号以及对话的内容适应于用户，（4）提供多任务学习和迁移学习技术，以使用（2）中获得的多文化语料库学习模型，并使会话代理适应每种文化，（5）提出基于心理学理论和评估研究的自适应AI系统的设计基础。

The key concept of PANORAMA is "user adaptive AI in the context of human-computer interaction". First, we will conduct research on user adaptivity of Artificial Intelligence embodied as a conversational agent. When people talk to other people, they change their verbal and nonverbal communication behaviors continuously according to those of the partner. Therefore, user adaptivity is an essential issue in improving human-agent interaction. Communication style is also different depending on the culture, and adapting the agent behaviors to a target culture is required in system localization. PANORAMA will tackle these problems with Machine Learning. However, a bottleneck of this approach is that annotating users’ nonverbal behaviors to create training data is time consuming. We will solve this problem by exploiting Explainable Artificial Intelligence (XAI) technique, through which labels predicted by the system are adapted based on the interaction with the user as an annotator. Thus, user adaptive AI enables to support users in creating multimodal corpus as well as improve human-agent interaction. Moreover, user adaptivity is considered in PANORAMA via psychological theories, in which user motivation will be investigated in one relevant use case (personalised motivational coaching for physical activity). Therefore, PANORAMA envisions a new research methodology for Machine-Learning-based culture-specific conversational agents by focusing on the concept of user adaptivity.PANORAMA aims to accomplish the following five research goals. (1) propose a user adaptive multimodal annotation tool based on XAI techniques, (2) exploit this tool to collect annotated multimodal corpora in three countries (France, Germany, and Japan), (3) propose models and methods for developing conversational agents with multi-level adaptation functionality, where nonverbal signals of the agent as well as the content of the dialogue are adapted to the user, (4) provide multitask learning and transfer learning techniques to learn models using the multi-cultural corpus obtained in (2) and adapt the conversational agent to each culture, and (5) propose the design basis of adaptive AI systems grounded in psychological theories and evaluation studies.