Zero-Shot Information Extraction for Clinical Meta-Analysis using Large Language Models
Zero-Shot Information Extraction for Clinical Meta-Analysis using Large Language Models
复制标题
DOI：
10.18653/v1/2023.bionlp-1.37
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
David Kartchner;Selvi Ramalingam;Irfan Al-Hussaini;Olivia Kronick;Cassie S. Mitchell
中科院分区：
文献类型：
--
作者：
David Kartchner;Selvi Ramalingam;Irfan Al-Hussaini;Olivia Kronick;Cassie S. Mitchell
随机临床试验（RCT）的荟萃分析在循证医学中起着至关重要的作用，但可能是劳动密集型和容易出错的。本研究探讨了使用大型语言模型来提高大规模随机临床试验（RCT）结果的聚合效率。我们进行了详细的比较，这些模型的性能在零杆基于随机对照试验的信息提取从一组不同的传统手动注释方法。我们分析了两个不同的荟萃分析的结果，目的是在慢性粒细胞白血病的癌症治疗药物警戒性的药物再利用。我们的研究结果表明，ChatGPT是两个演示任务的最佳模型，通常可以提取正确的信息，并确定文章中缺少所需的信息。我们还进行了系统的错误分析，记录在基于数据库的信息提取过程中遇到的各种错误类型的患病率。
Meta-analysis of randomized clinical trials (RCTs) plays a crucial role in evidence-based medicine but can be labor-intensive and error-prone. This study explores the use of large language models to enhance the efficiency of aggregating results from randomized clinical trials (RCTs) at scale. We perform a detailed comparison of the performance of these models in zero-shot prompt-based information extraction from a diverse set of RCTs to traditional manual annotation methods. We analyze the results for two different meta-analyses aimed at drug repurposing in cancer therapy pharmacovigilience in chronic myeloid leukemia. Our findings reveal that the best model for the two demonstrated tasks, ChatGPT can generally extract correct information and identify when the desired information is missing from an article. We additionally conduct a systematic error analysis, documenting the prevalence of diverse error types encountered during the process of prompt-based information extraction.