Using AI-generated suggestions from ChatGPT to optimize clinical decision support.
Using AI-generated suggestions from ChatGPT to optimize clinical decision support.
复制标题
使用ChatGPT的AI生成建议来优化临床决策支持。
DOI：
10.1093/jamia/ocad072
复制
发表时间：
2023-06-20
影响因子：
6.4
通讯作者：
Wright, Adam
中科院分区：
文献类型：
--
作者：
Liu, Siru;Wright, Aileen P.;Patterson, Barron L.;Wanderer, Jonathan P.;Turer, Robert W.;Nelson, Scott D.;McCoy, Allison B.;Sittig, Dean F.;Wright, Adam
确定ChatGPT是否可以生成用于改善临床决策支持（CDS）逻辑的有用建议，并评估与人工生成建议相比的非劣效性。我们向ChatGPT提供CDS逻辑的摘要，ChatGPT是一种使用大型语言模型的人工智能（AI）问答工具，并要求它生成建议。我们要求人类临床医生评审员审查人工智能生成的建议以及人类生成的建议，以改善相同的CDS警报，并对这些建议的有用性、接受度、相关性、理解、工作流程、偏见、倒置和冗余进行评级。五名临床医生分析了36个人工智能生成的建议和29个人工生成的建议，共7个警报。在调查中得分最高的20条建议中，有9条是由ChatGPT生成的。人工智能产生的建议被认为提供了独特的视角，并被评价为高度可理解和相关，具有中等实用性，低接受度，偏见，反转，冗余。人工智能生成的建议可能是优化CDS警报的重要补充部分，可以识别警报逻辑的潜在改进并支持其实施，甚至可以帮助专家制定自己的CDS改进建议。ChatGPT显示出巨大的潜力，可以使用大型语言模型和来自人类反馈的强化学习来改善CDS警报逻辑，并可能改善其他涉及复杂临床逻辑的医疗领域，这是开发高级学习健康系统的关键一步。
To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions. We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it to generate suggestions. We asked human clinician reviewers to review the AI-generated suggestions as well as human-generated suggestions for improving the same CDS alerts, and rate the suggestions for their usefulness, acceptance, relevance, understanding, workflow, bias, inversion, and redundancy. Five clinicians analyzed 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts. Of the 20 suggestions that scored highest in the survey, 9 were generated by ChatGPT. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy. AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.
登录
查看更多内容
DOI：
10.1093/jamia/ocac027
发表时间：
2022-05-11
影响因子：
6.4
作者：
McCoy, Allison B.;Russo, Elise M.;Johnson, Kevin B.;Addison, Bobby;Patel, Neal;Wanderer, Jonathan P.;Mize, Dara E.;Jackson, Jon G.;Reese, Thomas J.;Littlejohn, SyLinda;Patterson, Lorraine;French, Tina;Preston, Debbie;Rosenbury, Audra;Valdez, Charlie;Nelson, Scott D.;Aher, Chetan, V;Alrifai, Mhd Wael;Andrews, Jennifer;Cobb, Cheryl;Horst, Sara N.;Johnson, David P.;Knake, Lindsey A.;Lewis, Adam A.;Parks, Laura;Parr, Sharidan K.;Patel, Pratik;Patterson, Barron L.;Smith, Christine M.;Suszter, Krystle D.;Turer, Robert W.;Wilcox, Lyndy J.;Wright, Aileen P.;Wright, Adam
通讯作者：
Wright, Adam
影响因子：
--
作者：
Thomas Craig KJ;Fusco N;Lindsley K;Snowdon JL;Willis VC;Arriaga YE;Dankwa-Mullan I
通讯作者：
Dankwa-Mullan I
影响因子：
8
作者：
Daniels, Calvin C.;Burlison, Jonathan D.;Hoffman, James M.
通讯作者：
Hoffman, James M.
DOI：
10.1093/jamia/ocab292
发表时间：
2022-04-13
影响因子：
6.4
作者：
Liu, Siru;Kawamoto, Kensaku;Abdelrahman, Samir
通讯作者：
Abdelrahman, Samir
影响因子：
2.7
作者：
Reese, Thomas;Wright, Adam;Malone, Daniel
通讯作者：
Malone, Daniel