ERI: Developing a Trust-supporting Design Framework with Affect for Human-AI Collaboration

此工程研究启动(ERI)项目旨在了解在用户与工程设计的人工智能(AI)系统之间的协作期间，情感在发展信任中的作用，然后将为改善信任和协作结果的设计指导方针提供信息。现代人工智能系统已经变得比传统的自动化系统更具互动性和复杂性。然而，人类和人工智能系统之间的信任处于潜在危机中。虽然情感是信任发展的基本要素，并可以显著影响人们的协作倾向，但在人类-人工智能协作中尚未对其进行定量研究，也没有从系统设计属性的角度对其进行整体建模。为了弥补这一差距，本研究将：(1)考察情感在与人工智能支持的会话代理交互过程中对用户信任的影响；(2)调查对人工智能的信任如何影响人与人工智能的协作；(3)开发一个综合考虑情感和系统属性的价值模型。这项研究将促进信任发展的知识，重点放在情感和人工智能系统的设计上。这些知识将为设计框架提供信息，使人工智能系统能够被信任、接受和更多地利用，以增强人类完成复杂任务的能力。改善人类与人工智能的合作将提高人类的生活质量，并可能使弱势群体受益，例如不是早期采用技术的老年人，以及信任可能脆弱的远程医疗应用。该项目的总体目标是促进对人类-人工智能(AI)协作中信任发展本质的了解，以及工程设计人员如何增强对AI系统设计的信任，以改善协作结果。该研究考察了情感、系统性能和系统特征对协作设计任务中人工智能支持的对话代理中用户信任的影响，并调查了人与人工智能之间的信任如何影响设计结果。这项研究将使用修改后的情感和系统相关属性的联合分析，对用户的价值判断进行整体建模。结果将为创新的设计框架提供信息，该框架表明属性的相对重要性和对人类-人工智能协作有利的设计方向。这项研究有可能产生广泛的社会影响，因为它为如何改善和维持值得信赖的人类-人工智能合作提供了切实的指导方针，这种合作增强了人类完成认知要求高的任务的能力，可能特别有利于弱势群体。这项研究为网络界面和对话以外的更广泛的设计应用奠定了基础，并将提供虚拟环境和复杂系统中人-人工智能交互的变革性知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Engineering Research Initiation (ERI) project aims to understand the role of affect in developing trust during collaborations between users and Artificial Intelligence (AI) system for engineering design, which will then inform design guidelines for improving trust and collaboration outcomes. Modern AI systems have become more interactive and sophisticated than legacy automated systems. However, trust between humans and AI systems is in a potential crisis. While affect is an essential element of trust development and can significantly influence people’s tendency to collaborate, it has not been studied quantitatively in human-AI collaboration or modeled holistically with respect to system design attributes. To address this gap, this research will: (1) examine the influence of affect in user trust during interactions with an AI-powered conversational agent; (2) investigate how trust in AI influences human-AI collaboration; and (3) develop a value model that considers affect and system attributes comprehensively. This research will advance knowledge of trust development with an emphasis on affect and the design of AI systems. The knowledge will inform a design framework that enables AI systems to be trusted, accepted, and utilized more to augment human capabilities in completing complex tasks. Improved human-AI collaboration will enhance the quality of human life and potentially benefit disadvantaged populations, such as senior citizens, who are not early technology adopters, as well as telehealth applications, where trust can be tenuous. The overarching goal of this project is to advance knowledge about the nature of trust development in human-Artificial Intelligence (AI) collaboration and how engineering designers can enhance trust in the design of AI systems to improve collaboration outcomes. The research examines the effect of affect, system performance, and system characteristics on user trust in an AI-powered conversational agent for a collaborative design task, and investigates how human-AI trust influences design outcomes. The research will holistically model users’ value judgments using a modified conjoint analysis of both affect and system-related attributes. The results will inform an innovative design framework that indicates the relative importance of the attributes and favorable design directions for human-AI collaboration. The research has the potential for broad societal impact by providing tangible guidelines for how to improve and maintain trustworthy human-AI collaborations, which augment the human capability of completing cognitively demanding tasks and may particularly benefit disadvantaged populations. The research serves as the basis for a broader range of design applications beyond web interfaces and dialogues, and will provide transformative knowledge of human-AI interactions in virtual environments and for complex systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.