Natural language processing for automated quantification of bone metastases reported in free-text bone scintigraphy reports
Natural language processing for automated quantification of bone metastases reported in free-text bone scintigraphy reports
复制标题
DOI：
10.1080/0284186x.2020.1819563
复制
发表时间：
2020-09-11
期刊：
影响因子：
3.1
通讯作者：
Schwab, Joseph H.
中科院分区：
文献类型：
--
作者：
Groot, Olivier Q.;Bongers, Michiel E. R.;Schwab, Joseph H.
背景患者生成的电子健康数据的广泛使用为从自由文本医疗记录中自动提取临床特征带来了前所未有的机会。然而，为了临床和研究目的而处理这种丰富的数据资源，依赖于劳动密集型和潜在的容易出错的手动审查。本研究的目的是开发一种自然语言处理（NLP）算法，用于对接受骨转移手术的患者的骨肿瘤造影报告进行二元分类（单个转移与两个或两个以上转移）。材料和方法骨转移瘤手术患者的骨密度图报告由三名独立的评审员使用二元分类（单一转移与两个或两个以上转移）进行标记，以建立基础事实。使用分层的80：20分割来开发和测试极值梯度提升监督机器学习NLP算法。结果共纳入704例患者的704份自由文本骨显像报告，其中617例（88%）为多发性骨转移瘤。在未用于模型开发的独立测试集（n = 141）中，NLP算法实现了多发性骨转移分类的0.97 AUC-ROC（95%置信区间[CI]，0.92-0.99）和0.99 AUC-PRC（95% CI，0.99-0.99）。在阈值为0.90时，NLP算法正确识别了测试队列中124例多发性骨转移患者中的117例多发性骨转移（灵敏度0.94），并产生了3例假阳性（特异性0.82）。在相同的阈值下，NLP算法的阳性预测值为0.97，F1评分为0.96。结论NLP有潜力从骨科的自由文本放射学笔记中自动提取临床数据，从而优化临床图表审查的速度，准确性和一致性。在等待外部验证的情况下，本研究中开发的NLP算法可以作为一种手段来帮助研究人员处理大量数据。
Background The widespread use of electronic patient-generated health data has led to unprecedented opportunities for automated extraction of clinical features from free-text medical notes. However, processing this rich resource of data for clinical and research purposes, depends on labor-intensive and potentially error-prone manual review. The aim of this study was to develop a natural language processing (NLP) algorithm for binary classification (single metastasis versus two or more metastases) in bone scintigraphy reports of patients undergoing surgery for bone metastases. Material and methods Bone scintigraphy reports of patients undergoing surgery for bone metastases were labeled each by three independent reviewers using a binary classification (single metastasis versus two or more metastases) to establish a ground truth. A stratified 80:20 split was used to develop and test an extreme-gradient boosting supervised machine learning NLP algorithm. Results A total of 704 free-text bone scintigraphy reports from 704 patients were included in this study and 617 (88%) had multiple bone metastases. In the independent test set (n = 141) not used for model development, the NLP algorithm achieved an 0.97 AUC-ROC (95% confidence interval [CI], 0.92-0.99) for classification of multiple bone metastases and an 0.99 AUC-PRC (95% CI, 0.99-0.99). At a threshold of 0.90, NLP algorithm correctly identified multiple bone metastases in 117 of the 124 who had multiple bone metastases in the testing cohort (sensitivity 0.94) and yielded 3 false positives (specificity 0.82). At the same threshold, the NLP algorithm had a positive predictive value of 0.97 and F1-score of 0.96. Conclusions NLP has the potential to automate clinical data extraction from free text radiology notes in orthopedics, thereby optimizing the speed, accuracy, and consistency of clinical chart review. Pending external validation, the NLP algorithm developed in this study may be implemented as a means to aid researchers in tackling large amounts of data.