Trust and Explainable AI in Human-Machine Interaction

协作式人机交互（HMI）中的信任是一个双向过程。一个取决于用户对机器人/机器能力的感知，而第二个则考虑机器人对人类意图和目标的感知和信任。在这两种情况下，这都依赖于每个代理都有另一个实体的“心智理论”（ToM）。汤姆是一种隐含推断他人的信念、意图和目标的能力。这已被证明与信任有关，无论是在人和HMI系统中（例如Gaudiello等人，2011年；Zanatto等人出版）。最近的HMI研究提出了人工ToM的信任计算模型。例如，Cangelosi及其合作者（Vinanzi等人，2019；Patacchiola和Cangelosi，在审查中）基于概率机器学习方法（例如信念网络）为机器人开发了人工ToM模型。与此同时，可解释的人工智能（XAI）系统已经被提出用于透明的智能系统，特别是在健康信息学领域，但在机器人领域的应用很少（Anjomshoae等人，2019；Wachter等人，2017）。这个iCase博士项目旨在为人机交互开发新颖的、可解释的ToM模型。概率机器人ToM模型与可解释的人工智能方法的集成，通过添加“显式”ToM构建和更新组件，以补充现有的“隐式”意图读取模型，为提高协作HMI场景中的信任提供了机会。此外，在机器的决策过程中，可解释的AI交互可以允许交互代理修复他们的ToM，例如在不确定和模糊的情况下，以及在产生错误时。这些可解释的ToM模型有助于在柔性制造场景或与BAE系统相关的其他HMI场景中进行人机交互，以完成联合操作任务。这样的项目通过将不确定情况下的信任和ToM建模与可解释的AI联系起来，直接有助于HMI中的不确定性、模糊性和信任这一主题。

Trust in collaborative human-machine interaction (HMI) is a two-way process. One depends on the user's perception of the robot/machine's capabilities, whilst the second considers the robot's perception and trust of the human's intention and goals. In both cases, this relies on each agent having a "Theory of Mind" (ToM) of the other entity. A ToM is the capability to infer implicitly the belief, intention and goals of the other person. This has been shown to be linked to trust, in both people and HMI systems (e.g. Gaudiello et al. 2011; Zanatto et al. in press).Recent approaches in HMI have proposed computational models of artificial ToM for trust. For example, Cangelosi and collaborators (Vinanzi et al. 2019; Patacchiola and Cangelosi, in review) have developed artificial ToM models for robots based on probabilistic machine learning methods (e.g. belief network). In parallel, explainable AI (XAI) systems have been proposed for transparent intelligent systems, especially in the field of health informatics, but with only few applications to robotics (Anjomshoae et al. 2019; Wachter et al. 2017). This iCase PhD project aims at the development of novel, explainable ToM models for human-robot interaction. The integration of probabilistic robot ToM models with explainable AI methods offer the opportunity to improve trust in collaborative HMI scenarios by adding a component of "explicit" ToM building and update, to complement existing "implicit" models of intention reading. Moreover, explainable AI interaction on the machine's decision making process can allow the interacting agents to repair their ToM, e.g. in uncertain and vague situations, and when errors are produced. These explainable ToM models can contribute to human-cobot (collaborative robots) interaction for joint manipulation task within a flexible manufacturing scenario, or in other HMI scenarios relevant to BAE Systems. Such a project directly contributes to the topic on uncertainty, vagueness and trust in HMI, by linking trust and ToM modelling with explainable AI for uncertain situations.