Automated deidentification of radiology reports combining transformer and "hide in plain sight" rule-based methods
Automated deidentification of radiology reports combining transformer and "hide in plain sight" rule-based methods
复制标题
DOI：
10.1093/jamia/ocac219
复制
发表时间：
2022-11-23
影响因子：
6.4
通讯作者：
Langlotz, Curtis P.
中科院分区：
文献类型：
--
作者：
Chambon, Pierre J.;Wu, Christopher;Langlotz, Curtis P.
目的开发一种检测受保护健康信息（PHI）实体的放射学报告自动去识别管道，并将其替换为“隐藏在视线中”的真实替代品。在本回顾性研究中，收集了2019年11月至2020年11月期间收集的999份胸部x光片和CT报告，并在标记水平上对PHI进行了注释，并结合之前标记的3001份x光片和2193份医疗记录，形成了一个包含6193份文件的大型多机构跨领域数据集。来自一家知名机构和一家新机构的两个放射学测试集，以及2006年和2014年的i2b2测试集，作为评估集来估计模型的性能，并将其与之前发布的去识别工具进行比较。基于不同的训练数据集、微调方法和数据增强技术以及综合PHI生成算法，开发了几种PHI检测模型。使用精度、召回率和F1分数以及配对样本Wilcoxon测试等指标对这些模型进行比较。结果我们的最佳PHI检测模型在已知机构的放射学报告中达到97.9分，在新机构的放射学报告中达到99.6分，在2006年i2b2中达到99.5分，在2014年i2b2中达到98.9分。在一个已知机构的报告中，它在检测每个PHI跨度的核心时达到99.1的召回率。我们的模型优于所有测试集上的所有标识符，以及2014年2月2日数据上的人工标注器。它能够准确和自动地去识别放射学报告。结论基于变压器的去识别管道可以达到最先进的去识别放射报告和其他医学文件的性能。
Objective To develop an automated deidentification pipeline for radiology reports that detect protected health information (PHI) entities and replaces them with realistic surrogates "hiding in plain sight." Materials and Methods In this retrospective study, 999 chest X-ray and CT reports collected between November 2019 and November 2020 were annotated for PHI at the token level and combined with 3001 X-rays and 2193 medical notes previously labeled, forming a large multi-institutional and cross-domain dataset of 6193 documents. Two radiology test sets, from a known and a new institution, as well as i2b2 2006 and 2014 test sets, served as an evaluation set to estimate model performance and to compare it with previously released deidentification tools. Several PHI detection models were developed based on different training datasets, fine-tuning approaches and data augmentation techniques, and a synthetic PHI generation algorithm. These models were compared using metrics such as precision, recall and F1 score, as well as paired samples Wilcoxon tests. Results Our best PHI detection model achieves 97.9 F1 score on radiology reports from a known institution, 99.6 from a new institution, 99.5 on i2b2 2006, and 98.9 on i2b2 2014. On reports from a known institution, it achieves 99.1 recall of detecting the core of each PHI span. Discussion Our model outperforms all deidentifiers it was compared to on all test sets as well as human labelers on i2b2 2014 data. It enables accurate and automatic deidentification of radiology reports. Conclusions A transformer-based deidentification pipeline can achieve state-of-the-art performance for deidentifying radiology reports and other medical documents.