A vignette-based evaluation of ChatGPT's ability to provide appropriate and equitable medical advice across care contexts.
A vignette-based evaluation of ChatGPT's ability to provide appropriate and equitable medical advice across care contexts.
复制标题
DOI：
10.1038/s41598-023-45223-y
复制
发表时间：
2023-10-19
影响因子：
4.6
通讯作者：
Weissman, Gary E.
中科院分区：
文献类型：
--
作者：
Nastasi, Anthony J.;Courtright, Katherine R.;Halpern, Scott D.;Weissman, Gary E.
ChatGPT是一个在文本语料库上训练的大型语言模型，并在人类监督下得到加强。由于ChatGPT可以对复杂的问题提供类似人类的回答，因此它可以成为患者轻松获得医疗建议的来源。然而，它是否有能力适当和公平地回答医疗问题仍然是个未知数。我们向ChatGPT展示了96个寻求建议的小插曲，这些小插曲在临床背景、病史和社会特征方面各不相同。我们通过与指南的一致性、推荐类型和社会因素的考虑来分析临床适当性的反应。93例（97%）的反应是适当的，没有明确违反临床指南。针对咨询问题的建议完全不存在（N = 34，35%）、一般性（N = 18，18%）或具体性（N = 44，46%）。53人（55%）明确考虑了种族或保险状况等社会因素，在某些情况下改变了临床建议。ChatGPT在回答医疗问题时始终提供背景信息，但没有可靠地提供适当和个性化的医疗建议。
ChatGPT is a large language model trained on text corpora and reinforced with human supervision. Because ChatGPT can provide human-like responses to complex questions, it could become an easily accessible source of medical advice for patients. However, its ability to answer medical questions appropriately and equitably remains unknown. We presented ChatGPT with 96 advice-seeking vignettes that varied across clinical contexts, medical histories, and social characteristics. We analyzed responses for clinical appropriateness by concordance with guidelines, recommendation type, and consideration of social factors. Ninety-three (97%) responses were appropriate and did not explicitly violate clinical guidelines. Recommendations in response to advice-seeking questions were completely absent (N = 34, 35%), general (N = 18, 18%), or specific (N = 44, 46%). 53 (55%) explicitly considered social factors like race or insurance status, which in some cases changed clinical recommendations. ChatGPT consistently provided background information in response to medical questions but did not reliably offer appropriate and personalized medical advice.
登录
查看更多内容
影响因子：
18.2
作者：
Jaarsma, Tiny;Beattie, James M.;McMurray, John
通讯作者：
McMurray, John
DOI：
10.1161/cir.0000000000000625
发表时间：
2019-06-25
影响因子：
24
作者：
Grundy, Scott M.;Stone, Neil J.;Wijeysundera, Duminda N.
通讯作者：
Wijeysundera, Duminda N.
影响因子：
--
作者：
Freeman, Alexandra L J
通讯作者：
Freeman, Alexandra L J
DOI：
10.1371/journal.pdig.0000198
发表时间：
2023-02
期刊：
PLOS digital health
影响因子：
--
作者：
通讯作者：
--
影响因子：
4.5
作者：
Harris PA;Taylor R;Thielke R;Payne J;Gonzalez N;Conde JG
通讯作者：
Conde JG