Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study.
Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study.
复制标题
DOI：
10.2196/40384
复制
发表时间：
2022-08-30
影响因子：
7.4
通讯作者：
Das, Sudeshna
中科院分区：
文献类型：
--
作者：
Noori, Ayush;Magdamo, Colin;Liu, Xiao;Tyagi, Tanish;Li, Zhaozhi;Kondepudi, Akhil;Alabsi, Haitham;Rudmann, Emily;Wilcox, Douglas;Brenner, Laura;Robbins, Gregory K.;Moura, Lidia;Zafar, Sahar;Benson, Nicole M.;Hsu, John;Dickson, John R.;Serrano-Pozo, Alberto;Hyman, Bradley;Blacker, Deborah;Westover, M. Brandon;Mukerji, Shibani S.;Das, Sudeshna
关键词：
电子健康记录（EHR）具有大样本量和丰富的信息，为痴呆症研究提供了巨大的潜力，但目前的表型认知状态的方法是不可扩展的。本研究的目的是评估自然语言处理（NLP）驱动的半自动化注释是否可以提高表型认知状态图表审查的速度和评分者间可靠性。在这项诊断研究中，我们开发并评估了一种半自动化NLP驱动的注释工具（NAT），以促进认知状态的表型分析。临床专家使用NAT或传统的图表审查对马萨诸塞州总布里格姆（MGB）医疗保健中心的627名患者的认知状态进行了裁定。患者图表包含来自两个数据集的EHR数据：（1）从2017年1月1日到2018年12月31日的记录，来自MGB责任医疗组织的100名医疗保险受益人，以及（2）从COVID-19诊断前2年到COVID-19诊断日期的527名MGB患者。提取了相关时期的所有EHR数据;处理和总结了诊断代码、药物和实验室检查值;通过NLP管道处理了临床记录;开发了一个网络工具来呈现所有数据的综合视图。认知状态被评定为认知正常、认知受损或未确定。评估时间和评价者之间的协议NAT相比，人工图表审查的认知状态表型进行了评价。NAT裁定提供了更高的评分者间一致性（Cohen κ=0.89 vs κ=0.80）和显著的加速（时间差平均值1.4，SD 1.3分钟; P<0.001;比值中位数2.2，最小值-最大值0.4-20）。与手动病历审查的一致性中等（Cohen κ=0.67）。在与手动病历审查不一致的情况下，NAT裁定能够产生具有更广泛临床共识的评估，这是由于其对突出显示的相关信息和半自动NLP功能的综合视图。与手动图表审查相比，NAT裁定提高了表型认知状态的速度和评估者间可靠性。这项研究强调了一种基于NLP的临床裁定方法的潜力，可以从EHR中建立大规模的痴呆研究队列。
Electronic health records (EHRs) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are not scalable. The aim of this study was to evaluate whether natural language processing (NLP)–powered semiautomated annotation can improve the speed and interrater reliability of chart reviews for phenotyping cognitive status. In this diagnostic study, we developed and evaluated a semiautomated NLP-powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) health care, using NAT or traditional chart reviews. Patient charts contained EHR data from two data sets: (1) records from January 1, 2017, to December 31, 2018, for 100 Medicare beneficiaries from the MGB Accountable Care Organization and (2) records from 2 years prior to COVID-19 diagnosis to the date of COVID-19 diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized; clinical notes were processed through an NLP pipeline; and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated. NAT adjudication provided higher interrater agreement (Cohen κ=0.89 vs κ=0.80) and significant speed up (time difference mean 1.4, SD 1.3 minutes; P<.001; ratio median 2.2, min-max 0.4-20) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen κ=0.67). In the cases that exhibited disagreement with manual chart reviews, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semiautomated NLP features. NAT adjudication improves the speed and interrater reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP-based clinically adjudicated method to build large-scale dementia research cohorts from EHRs.
DOI：
10.5334/egems.287
发表时间：
2019-01-24
期刊：
EGEMS (Washington, DC)
影响因子：
--
作者：
Sendak, Mark;Gao, Michael;Balu, Suresh
通讯作者：
Balu, Suresh
影响因子：
14.8
作者：
Zhang, Yichi;Cai, Tianrun;Liao, Katherine P.
通讯作者：
Liao, Katherine P.
影响因子：
3.3
作者：
Wong, Jenna;Murray Horwitz, Mara;Toh, Sengwee
通讯作者：
Toh, Sengwee