Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies
Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies
复制标题
迈向人类人工智能决策科学：实证人类受试者研究中的设计空间概述
DOI：
10.1145/3593013.3594087
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Tan, Chenhao
中科院分区：
文献类型：
--
作者：
Lai, Vivian;Chen, Chacha;Smith-Renner, Alison;Liao, Q. Vera;Tan, Chenhao
人工智能系统由于其日益强大的预测性能而被广泛采用。然而，在诸如刑事司法和医疗保健等高风险领域，由于安全、道德和法律的考虑，全自动化通常是不可取的，而全手动方法可能不准确且耗时。因此，研究界越来越有兴趣通过人工智能辅助来增强人类决策。除了为此目的开发人工智能技术外，人工智能决策的新兴领域必须采用经验方法，以形成对人类如何与人工智能互动和合作做出决策的基本理解。为了邀请并帮助构建理解和改善人类人工智能决策科学的研究工作，我们调查了最近关于这一主题的实证人类受试者研究文献。我们从三个重要方面总结了100多篇论文中的研究设计选择：（1）决策任务，（2）AI辅助元素，（3）评估指标。对于每个方面，我们总结了当前的趋势，讨论了该领域当前实践中的差距，并为未来的研究提出了建议。我们的工作强调了开发通用框架的必要性，以解释人类人工智能决策的设计和研究空间，以便研究人员可以在研究设计中做出严格的选择，研究界可以在彼此的工作基础上建立并产生可推广的科学知识。我们还希望这项工作将成为HCI和AI社区共同努力的桥梁，共同塑造人类-AI决策的经验科学和计算技术。
AI systems are adopted in numerous domains due to their increasingly strong predictive performance. However, in high-stakes domains such as criminal justice and healthcare, full automation is often not desirable due to safety, ethical, and legal concerns, yet fully manual approaches can be inaccurate and time-consuming. As a result, there is growing interest in the research community to augment human decision making with AI assistance. Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions. To invite and help structure research efforts towards a science of understanding and improving human-AI decision making, we survey recent literature of empirical human-subject studies on this topic. We summarize the study design choices made in over 100 papers in three important aspects: (1) decision tasks, (2) AI assistance elements, and (3) evaluation metrics. For each aspect, we summarize current trends, discuss gaps in current practices of the field, and make a list of recommendations for future research. Our work highlights the need to develop common frameworks to account for the design and research spaces of human-AI decision making, so that researchers can make rigorous choices in study design, and the research community can build on each other’s work and produce generalizable scientific knowledge. We also hope this work will serve as a bridge for HCI and AI communities to work together to mutually shape the empirical science and computational technologies for human-AI decision making.
登录
查看更多内容
解释是否能让 VQA 模型对人类来说更具可预测性？
DOI：
10.18653/v1/d18-1128
发表时间：
2018
期刊：
Advances in neural information processing systems
影响因子：
--
作者：
Arjun Chandrasekaran;Viraj Prabhu;Deshraj Yadav;Prithvijit Chattopadhyay;Devi Parikh
通讯作者：
Devi Parikh
DOI：
--
发表时间：
2018-11
期刊：
ArXiv
影响因子：
--
作者：
Aaron Springer;S. Whittaker
通讯作者：
Aaron Springer;S. Whittaker
DOI：
10.1145/3411764.3445562
发表时间：
2021-05
期刊：
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
影响因子：
--
作者：
Zhuoran Lu;Ming Yin
通讯作者：
Zhuoran Lu;Ming Yin
信息呈现对机器学习预测的公平性感知的影响
DOI：
--
发表时间：
2021
期刊：
International Conference on Human Factors in Computing Systems
影响因子：
--
作者：
Niels van Berkel;Jorge Gonçalves;D. Russo;S. Hosio;M. Skov
通讯作者：
M. Skov
影响因子：
0.9
作者：
Amershi, Saleema;Cakmak, Maya;Kulesza, Todd
通讯作者：
Kulesza, Todd