Retrospective comparison of traditional and artificial intelligence-based heart failure phenotyping in a US health system to enable real-world evidence.
Retrospective comparison of traditional and artificial intelligence-based heart failure phenotyping in a US health system to enable real-world evidence.
复制标题
DOI：
10.1136/bmjopen-2023-073178
复制
发表时间：
2023-08-09
期刊：
影响因子：
2.9
通讯作者：
Gluckman, Ty J.
中科院分区：
文献类型：
--
作者：
Garan, Arthur Reshad;Monda, Keri L.;Dent-Acosta, Ricardo E.;Riskin, Daniel J.;Gluckman, Ty J.
定量评价心力衰竭（HF）真实世界证据（RWE）基础数据的质量。使用传统（即，应用于结构化电子健康记录（EHR）数据的结构化查询语言）和高级（即，应用于非结构化EHR数据的人工智能（AI））RWE方法，对识别HF患者和表型信息的准确性进行了回顾性比较。每种方法的性能进行了测量的精确度和召回率（F1得分）的调和平均值，使用手动注释的医疗记录作为参考标准。2015年至2019年期间，来自北美大型学术医疗保健系统的EHR数据，预计覆盖约50万名患者。1155名年龄在18-85岁之间的患者进行了4288次就诊，其中472名患者被确定患有HF。HF和相关概念，如合并症、左心室射血分数和选定的药物。传统方法和高级方法的19个HF特定概念的平均F1评分分别为49.0%和94.1%（对于所有有可用数据的概念，p<0.001）。两种方法之间F1评分的绝对差异为45.1%（使用高级方法的F1评分相对增加98.1%）。先进的方法实现了HF存在、表型和相关合并症的上级F1评分。一些表型，如射血分数保留的HF，显示出基于应用技术的提取准确性的显着差异，单独使用自然语言处理（NLP）时的F1得分为4.9%，使用NLP加基于AI的推理时的F1得分为91.0%。传统的RWE生成方法导致HF患者的数据质量较低。虽然先进的方法显示出很高的准确性，但结果因提取技术而异。对于未来的研究，可能需要先进的方法和准确性测量，以确保数据适用于目的。
Quantitatively evaluate the quality of data underlying real-world evidence (RWE) in heart failure (HF). Retrospective comparison of accuracy in identifying patients with HF and phenotypic information was made using traditional (ie, structured query language applied to structured electronic health record (EHR) data) and advanced (ie, artificial intelligence (AI) applied to unstructured EHR data) RWE approaches. The performance of each approach was measured by the harmonic mean of precision and recall (F1 score) using manual annotation of medical records as a reference standard. EHR data from a large academic healthcare system in North America between 2015 and 2019, with an expected catchment of approximately 5 00 000 patients. 4288 encounters for 1155 patients aged 18–85 years, with 472 patients identified as having HF. HF and associated concepts, such as comorbidities, left ventricular ejection fraction, and selected medications. The average F1 scores across 19 HF-specific concepts were 49.0% and 94.1% for the traditional and advanced approaches, respectively (p<0.001 for all concepts with available data). The absolute difference in F1 score between approaches was 45.1% (98.1% relative increase in F1 score using the advanced approach). The advanced approach achieved superior F1 scores for HF presence, phenotype and associated comorbidities. Some phenotypes, such as HF with preserved ejection fraction, revealed dramatic differences in extraction accuracy based on technology applied, with a 4.9% F1 score when using natural language processing (NLP) alone and a 91.0% F1 score when using NLP plus AI-based inference. A traditional RWE generation approach resulted in low data quality in patients with HF. While an advanced approach demonstrated high accuracy, the results varied dramatically based on extraction techniques. For future studies, advanced approaches and accuracy measurement may be required to ensure data are fit-for-purpose.
登录
查看更多内容
影响因子：
4.3
作者：
Cunningham H;Tablan V;Roberts A;Bontcheva K
通讯作者：
Bontcheva K
DOI：
10.1136/amiajnl-2011-000557
发表时间：
2012-07
期刊：
Journal of the American Medical Informatics Association : JAMIA
影响因子：
--
作者：
Parsons A;McCullough C;Wang J;Shih S
通讯作者：
Shih S
影响因子：
24
作者：
Blecker S;Katz SD;Horwitz LI;Kuperman G;Park H;Gold A;Sontag D
通讯作者：
Sontag D
DOI：
10.1093/ehjqcco/qcab070
发表时间：
2022-10-26
影响因子：
5.2
作者：
Lim, Yvonne Mei Fong;Molnar, Megan;Vaartjes, Ilonca;Savarese, Gianluigi;Eijkemans, Marinus J. C.;Uijl, Alicia;Vradi, Eleni;Suzart-Woischnik, Kiliana;Brugts, Jasper J.;Brunner-La Rocca, Hans-Peter;Blanc-Guillemaud, Vanessa;Couvelard, Fabrice;Baudier, Claire;Dyszynski, Tomasz;Waechter, Sandra;Lund, Lars H.;Hoes, Arno W.;Tyl, Benoit;Asselbergs, Folkert W.;Gerlinger, Christoph;Grobbee, Diederick E.;Cronin, Maureen;Koudstaal, Stefan
通讯作者：
Koudstaal, Stefan
DOI：
10.1007/s12265-015-9644-2
发表时间：
2015-11-01
影响因子：
3.4
作者：
Bielinski, Suzette J.;Pathak, Jyotishman;Roger, Veronique L.
通讯作者：
Roger, Veronique L.