Natural Language Processing to identify pneumonia from radiology reports
Natural Language Processing to identify pneumonia from radiology reports
复制标题
DOI：
10.1002/pds.3418
复制
发表时间：
2013-08-01
影响因子：
2.6
通讯作者：
Chapman, Wendy W.
中科院分区：
文献类型：
--
作者：
Dublin, Sascha;Baldwin, Eric;Chapman, Wendy W.
本研究旨在开发自然语言处理（NLP）方法来补充人工结果验证，特别是验证胸片报告中的肺炎病例。方法我们训练了一个NLP系统，ONYX，使用儿童和成人的x线片报告，这些报告以前是手工审查的。然后我们在5000个报告的测试集上评估了它的有效性。我们的目标是大幅减少人工审查，而不是完全取代它，因此，我们将报告分类如下：(1)与肺炎一致；(2)与肺炎不符的；或者(3)由于复杂的特性需要人工审查。我们开发了定制的流程，以优化准确性或最小化人工审查。使用逻辑回归，我们联合建模ONYX的敏感性和特异性与患者年龄、合并症和护理环境的关系。假设源数据中肺炎流行，我们估计阳性和阴性预测值（PPV和NPV）。结果ONYX为准确性量身定制，确定25%的报告需要人工审查（34%的真肺炎和18%的非肺炎）。其余ONYX的敏感性为92% (95% CI 90-93%)，特异性为87% (86-88%)，PPV为74% (72-76%)，NPV为96%（96-97%）。为了最小化人工审查，ONYX将12%归类为需要人工审查。其余的ONYX敏感性75%(72-77%)，特异性95% (94-96%),PPV 86% (83-88%), NPV 91%（90-91%）。结论：对于肺炎验证，ONYX可以替代近90%的人工审核，同时保持低到中等的误分类率。它可以根据不同的结果和研究需求进行调整，因此值得在其他环境中进行探索。版权所有：John Wiley & Sons， Ltd
Purpose This study aimed to develop Natural Language Processing (NLP) approaches to supplement manual outcome validation, specifically to validate pneumonia cases from chest radiograph reports.Methods We trained one NLP system, ONYX, using radiograph reports from children and adults that were previously manually reviewed. We then assessed its validity on a test set of 5000 reports. We aimed to substantially decrease manual review, not replace it entirely, and so, we classified reports as follows: (1) consistent with pneumonia; (2) inconsistent with pneumonia; or (3) requiring manual review because of complex features. We developed processes tailored either to optimize accuracy or to minimize manual review. Using logistic regression, we jointly modeled sensitivity and specificity of ONYX in relation to patient age, comorbidity, and care setting. We estimated positive and negative predictive value (PPV and NPV) assuming pneumonia prevalence in the source data.Results Tailored for accuracy, ONYX identified 25% of reports as requiring manual review (34% of true pneumonias and 18% of non-pneumonias). For the remainder, ONYX's sensitivity was 92% (95% CI 90-93%), specificity 87% (86-88%), PPV 74% (72-76%), and NPV 96% (96-97%). Tailored to minimize manual review, ONYX classified 12% as needing manual review. For the remainder, ONYX had sensitivity 75% (72-77%), specificity 95% (94-96%), PPV 86% (83-88%), and NPV 91% (90-91%).Conclusions For pneumonia validation, ONYX can replace almost 90% of manual review while maintaining low to moderate misclassification rates. It can be tailored for different outcomes and study needs and thus warrants exploration in other settings. Copyright (C) 2013 John Wiley & Sons, Ltd.