On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications
On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications
复制标题
DOI：
10.1145/3531066
复制
发表时间：
2022-04
影响因子：
3.4
通讯作者：
Mahsan Nourani;Chiradeep Roy;Jeremy E. Block;Donald R. Honeycutt;Tahrima Rahman;E. Ragan;Vibhav Gogate
中科院分区：
文献类型：
--
作者：
Mahsan Nourani;Chiradeep Roy;Jeremy E. Block;Donald R. Honeycutt;Tahrima Rahman;E. Ragan;Vibhav Gogate
虽然可解释的人工智能（XAI）旨在通过改善模型透明度和心理模型形成来改善人类协作的决策，但与人类用户相关的专家因素可能会以系统设计师在本文中的方式引起挑战。展示了用户研究用户最初与智能系统互动以及的作用时，锚定偏差如何可能影响心理模型形成在使用烹饪领域的视频活动识别工具来解决此偏见时，我们要求参与者验证是否遵循一套厨房政策，每个政策都集中在弱点或力量上。测试我们的假设的解释的存在表明，那些早期观察到系统优势的人更容易自动化偏见该系统的印象虽然建立了系统能力的更准确的心理模型，但是那些遇到弱点的人却大大减少了错误，因为他们倾向于更多地依靠自己，而他们也低估了模型能力通过这些发现和类似的现有工作激发了模型的负面印象，我们对用户过去经验的概念模型进行了检查基于使用时间的XAI系统的背景，经验和人为因素。
While EXplainable Artificial Intelligence (XAI) approaches aim to improve human-AI collaborative decision-making by improving model transparency and mental model formations, experiential factors associated with human users can cause challenges in ways system designers do not anticipate. In this article, we first showcase a user study on how anchoring bias can potentially affect mental model formations when users initially interact with an intelligent system and the role of explanations in addressing this bias. Using a video activity recognition tool in cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. However, those who encountered weaknesses earlier made significantly fewer errors, since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Motivated by these findings and similar existing work, we formalize and present a conceptual model of user’s past experiences that examine the relations between user’s backgrounds, experiences, and human factors in XAI systems based on usage time. Our work presents strong findings and implications, aiming to raise the awareness of AI designers toward biases associated with user impressions and backgrounds.