Trust criteria for artificial intelligence in health: normative and epistemic considerations.
Trust criteria for artificial intelligence in health: normative and epistemic considerations.
复制标题
健康人工智能的信任标准：规范和认知考虑。
DOI：
10.1136/jme-2023-109338
复制
发表时间：
2023
影响因子：
4.1
通讯作者：
Blumenthal-Barby,Jennifer
中科院分区：
文献类型：
--
作者：
Kostick-Quenet,Kristin;Lang,BenjaminH;Smith,Jared;Hurley,Meghan;Blumenthal-Barby,Jennifer
人工智能和机器学习（AI/ML）在医疗保健领域的快速发展提出了一个紧迫的问题，即用户应该在多大程度上信任AI/ML系统，特别是在高风险的临床决策方面。确保用户信任正确校准到工具的计算能力和限制具有实际和伦理意义，因为过度信任或信任不足会影响对算法工具的过度依赖或依赖不足，对患者安全和健康结果具有重大影响。因此，重要的是要更好地了解利益相关者，设置，工具和用例之间的信任标准的变化如何影响在真实的设置中使用AI/ML工具的方法。作为一项为期5年的多机构医疗保健研究和质量机构资助的研究的一部分，我们确定了生存预测算法的信任标准，旨在支持左心室辅助装置治疗的临床决策，使用半结构化访谈（n = 40）与患者和医生，通过主题分析进行分析。研究结果表明，医生和患者对信任有着相似的经验考虑，主要是认识论性质，专注于AI/ML估计的准确性和有效性。信任评估考虑了训练数据的性质、完整性和相关性，而不是算法本身的计算性质，这表明需要区分"源"和"功能"的可解释性。在较小程度上，信任标准也是关系性的（来自他人的认可），有时基于个人信仰和经验。我们讨论了使用AI/ML促进适当和负责任的信任校准对临床决策的影响。
Rapid advancements in artificial intelligence and machine learning (AI/ML) in healthcare raise pressing questions about how much users should trust AI/ML systems, particularly for high stakes clinical decision-making. Ensuring that user trust is properly calibrated to a tool’s computational capacities and limitations has both practical and ethical implications, given that overtrust or undertrust can influence over-reliance or under-reliance on algorithmic tools, with significant implications for patient safety and health outcomes. It is, thus, important to better understand how variability in trust criteria across stakeholders, settings, tools and use cases may influence approaches to using AI/ML tools in real settings. As part of a 5-year, multi-institutional Agency for Health Care Research and Quality-funded study, we identify trust criteria for a survival prediction algorithm intended to support clinical decision-making for left ventricular assist device therapy, using semistructured interviews (n=40) with patients and physicians, analysed via thematic analysis. Findings suggest that physicians and patients share similar empirical considerations for trust, which were primarilyepistemicin nature, focused on accuracy and validity of AI/ML estimates. Trust evaluations considered the nature, integrity and relevance of training data rather than the computational nature of algorithms themselves, suggesting a need to distinguish ‘source’ from ‘functional’ explainability. To a lesser extent, trust criteria were also relational (endorsement from others) and sometimes based on personal beliefs and experience. We discuss implications for promoting appropriate and responsible trust calibration for clinical decision-making use AI/ML.