Trust and Explainable AI in Human-Machine Interaction

协作式人机交互（HMI）中的信任是一个双向过程。一个取决于用户对机器人/机器能力的感知，而第二个则考虑机器人对人类意图和目标的感知和信任。在这两种情况下，这都依赖于每个代理都有另一个实体的“心理理论”（ToM）。心理理论是一种能够含蓄地推断出他人的信念、意图和目标的能力。这已经被证明与人和HMI系统中的信任有关（例如，Gaudiello等人，2011; Zanatto等人，在出版中）。HMI中的最新方法已经提出了用于信任的人工ToM的计算模型。例如，Cangelosi和合作者（Vinanzi et al. 2019; Patacchiola and Cangelosi，in review）已经基于概率机器学习方法（例如信念网络）为机器人开发了人工ToM模型。与此同时，可解释人工智能（XAI）系统已被提出用于透明智能系统，特别是在健康信息学领域，但在机器人领域的应用很少（Anjomshoae et al. 2019; Wachter et al. 2017）。这个iCase博士项目旨在为人机交互开发新颖，可解释的ToM模型。概率机器人ToM模型与可解释的AI方法的集成提供了通过添加“显式”ToM构建和更新的组件来提高协作HMI场景中的信任的机会，以补充现有的“隐式”意图阅读模型。此外，机器决策过程中可解释的人工智能交互可以允许交互代理修复其ToM，例如在不确定和模糊的情况下，以及产生错误时。这些可解释的ToM模型可以有助于在柔性制造场景中或与BAE系统相关的其他HMI场景中进行联合操作任务的人-协作机器人（协作机器人）交互。这样一个项目直接有助于HMI中的不确定性，准确性和信任的主题，通过将信任和ToM建模与不确定情况下的可解释AI联系起来。

Trust in collaborative human-machine interaction (HMI) is a two-way process. One depends on the user's perception of the robot/machine's capabilities, whilst the second considers the robot's perception and trust of the human's intention and goals. In both cases, this relies on each agent having a "Theory of Mind" (ToM) of the other entity. A ToM is the capability to infer implicitly the belief, intention and goals of the other person. This has been shown to be linked to trust, in both people and HMI systems (e.g. Gaudiello et al. 2011; Zanatto et al. in press).Recent approaches in HMI have proposed computational models of artificial ToM for trust. For example, Cangelosi and collaborators (Vinanzi et al. 2019; Patacchiola and Cangelosi, in review) have developed artificial ToM models for robots based on probabilistic machine learning methods (e.g. belief network). In parallel, explainable AI (XAI) systems have been proposed for transparent intelligent systems, especially in the field of health informatics, but with only few applications to robotics (Anjomshoae et al. 2019; Wachter et al. 2017). This iCase PhD project aims at the development of novel, explainable ToM models for human-robot interaction. The integration of probabilistic robot ToM models with explainable AI methods offer the opportunity to improve trust in collaborative HMI scenarios by adding a component of "explicit" ToM building and update, to complement existing "implicit" models of intention reading. Moreover, explainable AI interaction on the machine's decision making process can allow the interacting agents to repair their ToM, e.g. in uncertain and vague situations, and when errors are produced. These explainable ToM models can contribute to human-cobot (collaborative robots) interaction for joint manipulation task within a flexible manufacturing scenario, or in other HMI scenarios relevant to BAE Systems. Such a project directly contributes to the topic on uncertainty, vagueness and trust in HMI, by linking trust and ToM modelling with explainable AI for uncertain situations.