Investigating Trust in Human-Machine Learning Collaboration: A Pilot Study on Estimating Public Anxiety from Speech
Investigating Trust in Human-Machine Learning Collaboration: A Pilot Study on Estimating Public Anxiety from Speech
复制标题
调查人机学习协作中的信任：从语音估计公众焦虑的试点研究
DOI：
10.1145/3462244.3479926
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Chaspari, Theodora
中科院分区：
文献类型：
--
作者：
Tutul, Abdullah Aman;Nirjhar, Ehsanul Haque;Chaspari, Theodora
信任是人类与日益复杂的人工智能（AI）系统之间建立有效协作关系的关键因素。在这里，我们在人类-人工智能伙伴关系的背景下研究了对人工智能的信任，该伙伴关系涉及一项联合决策任务，即根据语音信号估计公众演讲焦虑水平。人工智能系统由一个可解释的机器学习（ML）算法组成，该算法将声学特征作为输入，并输出对公开演讲焦虑水平的估计，对每个语音样本决策的最重要特征的局部解释，以及对整体数据最重要特征的全局解释。我们分析了人工智能和具有心理科学背景的人类注释者之间的交互，并通过注释者与人工智能模型的一致性和注释者的自我报告来衡量随着时间的推移的信任。我们进一步研究了与人类注释者和ML算法的特征相关的信任因素。结果表明，人工智能的信任度取决于注释者的开放程度和输入特征的重要程度。这项研究的结果可以为设计解决方案提供指导，这些解决方案可以在人类-人工智能协作任务中正确校准人类对人工智能的信任。
Trust is a key element in the development of effective collaborative relationships between humans and increasingly complex artificial intelligence (AI) systems. Here, we examine trust in AI in the context of a human-AI partnership that involves a joint decision making task for estimating levels of public speaking anxiety based on speech signals. The AI system is comprised of an explainable machine learning (ML) algorithm, that takes acoustic characteristics as input and outputs the estimate of public speaking anxiety levels, a local explanation about the most important features that contributed to the decision of each speech sample, and a global explanation about the most important features for the data overall. We analyze interactions between AI and human annotators with background in psychological sciences, and measure trust over time via the annotators’ agreement with the AI model and the annotators’ self-reports. We further examine factors of trust that are related to the characteristics of the human annotator and the ML algorithm. Results indicate that trust in AI depends on the openness level of the annotator and the importance level of input features. Findings from this study can provide guidelines to designing solutions that properly calibrate human trust in AI in collaborative human-AI tasks.
登录
查看更多内容
探索现实生活和虚拟演讲中公开演讲焦虑的个体差异
影响因子：
11.2
作者：
Megha Yadav;N. Sakib;E. Nirjhar;Kexin Feng;A. Behzadan;Theodora Chaspari
通讯作者：
Theodora Chaspari
人工智能解释对人类对面向患者的人工智能驱动的医疗系统的看法的影响
影响因子：
5.3
作者：
Zhan Zhang;Y. Genc;Dakuo Wang;M. Ahsen;Xiangmin Fan
通讯作者：
Xiangmin Fan
高风险场景下对机器人的过度信任
DOI：
--
发表时间：
2018
期刊：
AAAI/ACM Conference on AI, Ethics, and Society
影响因子：
--
作者：
Jin Xu
通讯作者：
Jin Xu
影响因子：
30.8
作者：
Thorsen-Meyer, Hans-Christian;Nielsen, Annelaura B.;Perner, Anders
通讯作者：
Perner, Anders
DOI：
10.1145/3397481.3450650
发表时间：
2021
期刊：
Proceedings of the 26th International Conference on Intelligent User Interfaces
影响因子：
--
作者：
Wang, Xinru;Yin, Ming
通讯作者：
Yin, Ming