Regional Oncology Research Center (LLMs for Unstructured Data Extraction)

摘要 人工智能(AI)有可能通过利用临床数据来推进医疗保健的革命性变革 研究和改进肿瘤学实践。在自由文本的病理报告中，关于初级 癌症诊断和进化的分子特征嵌入其中。提取和解释这些信息 准确判断肿瘤分期对预后和指导临床起着决定性作用。 管理层。尽管自然语言处理(NLP)技术已经被应用于提取关注的 来自病理报告的信息，仍然需要适应性的、可概括的和可解释的策略 加强临床数据的抽象化。为了满足这一需求，我们提出了一种多学科的方法来开发 一条一体化的临床信息提取管道。这项工作旨在改进、评估和增强 利用大型语言从病理报告中提取病理诊断的相关特征 模特们。 我们的研究设计包括几个步骤。首先，我们将建立一个多样化和公平的患者队列 从我们的癌症注册中心收集自由文本的病理报告，以及从 约翰霍普金斯大学医学院精密医学分析平台(PMAP)数据共享。接下来，我们 将利用信息提取平台从报告中识别病理特征。这个平台将 使用Microsoft提供的一套型号，包括类似BERT的型号、GPT-3.5和GPT-4，具体而言 设计用于识别关键的癌症属性。随后，我们将评估各个模型的输出 使用Casper交互模型开发框架，通过以下方式增强和提炼结果 启发式和监管不力。增强的模型输出将通过基于Web的用户呈现 界面，允许专家馆长提供进一步的投入。然后我们将比较每种方法的有效性 Casper增广模型及其所衍生的病理特征与已建立的金标准 来自癌症登记处的注释。最后，我们将增强基于GPT的语言模型 评估、管理和比较过程，采用及时的工程技术进行改进 性能和减轻偏见。

Abstract Artificial intelligence (AI) has the potential to revolutionize healthcare by leveraging clinical data to advance research and improve oncology practice. Within free-text pathology reports, crucial information about primary cancer diagnoses and evolving molecular features is embedded. Extracting and interpreting this information accurately is essential for determining cancer stage, which plays a decisive role in prognosis and guiding clinical management. Although natural language processing (NLP) techniques have been applied to extract focused information from pathology reports, there is still a need for adaptable, generalizable, and interpretable strategies to enhance clinical data abstraction. To address this need, we propose a multidisciplinary approach to develop an integrative clinical information extraction pipeline. This work aims to improve, assess, and enhance the abstraction of relevant features of pathological diagnosis from pathology reports by leveraging large language models. Our research design involves several steps. First, we will establish a diverse and equitable cohort of patients from our Cancer Registry and collect free-text pathology reports, along with structured clinical data obtained from the Johns Hopkins School of Medicine Precision Medicine Analytics Platform (PMAP) Data Commons. Next, we will employ an information extraction platform to identify pathological features from the reports. This platform will utilize a suite of models, including BERT-like models, GPT-3.5, and GPT-4, provided by Microsoft, specifically designed for identifying key cancer attributes. Subsequently, we will evaluate the output of individual models using the CASPER interactive model development framework, enhancing and refining the results through heuristics and weak supervision. The augmented model output will be presented through a web-based user interface, allowing expert curators to provide further input. We will then compare the effectiveness of each CASPER-augmented model and its derived pathological features against the established gold standard annotations from the Cancer Registry. Finally, we will enhance the GPT-based language models based on the assessment, curation, and comparison process, employing prompt engineering techniques to improve performance and mitigate bias.