Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
复制标题
DOI：
10.1145/3411764.3445562
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Zhuoran Lu;Ming Yin
中科院分区：
文献类型：
--
作者：
Zhuoran Lu;Ming Yin
本文讨论了人工智能辅助决策中一个未被充分探索的问题：当支持决策辅助的机器学习模型的客观性能信息缺失或稀缺时，人们如何决定他们对该模型的依赖？通过三个随机实验，我们探索了当性能反馈有限时，人们可以用来调整他们对机器学习模型的依赖的启发式方法。我们发现，如果人们没有收到关于模型绩效的信息，那么人们与模型在决策任务上的一致程度会显著影响人们对模型的依赖，但这种影响会在聚合水平的模型绩效信息可用后发生变化。此外，高自信的人类模型协议对人们对模型的依赖的影响被人们在不同意模型的情况下的信心所缓和。我们讨论了这些启发式方法的潜在风险，并为促进对人工智能的适当依赖提供了设计启示。
This paper addresses an under-explored problem of AI-assisted decision-making: when objective performance information of the machine learning model underlying a decision aid is absent or scarce, how do people decide their reliance on the model? Through three randomized experiments, we explore the heuristics people may use to adjust their reliance on machine learning models when performance feedback is limited. We find that the level of agreement between people and a model on decision-making tasks that people have high confidence in significantly affects reliance on the model if people receive no information about the model’s performance, but this impact will change after aggregate-level model performance information becomes available. Furthermore, the influence of high confidence human-model agreement on people’s reliance on a model is moderated by people’s confidence in cases where they disagree with the model. We discuss potential risks of these heuristics, and provide design implications on promoting appropriate reliance on AI.