Temporal bone radiology report classification using open source machine learning and natural langue processing libraries.
Temporal bone radiology report classification using open source machine learning and natural langue processing libraries.
复制标题
DOI：
10.1186/s12911-016-0306-3
复制
发表时间：
2016-06-06
影响因子：
3.5
通讯作者：
Crenshaw EB 3rd
中科院分区：
文献类型：
--
作者：
Masino AJ;Grundmeier RW;Pennington JW;Germiller JA;Crenshaw EB 3rd
放射学报告是生物医学研究的丰富资源。在使用之前，训练有素的专家必须手动审查报告，以确定不连续的结果。听力和遗传数据库(AudGenDB)是一个公共的、未识别的研究数据库，包含16,000多份放射学报告。由于这些报告没有标注，因此很难选择那些具有特定异常的报告。我们使用人在回路中的机器学习方法和开源库实现了分类管道，以使用四个异常区域标签中的一个或多个来标记报告：内、中、外和乳突，指示在指定的耳朵区域中存在异常。经过培训的抽象员标记了从AudGenDB获得的放射学报告，形成了一个黄金标准。这些被分为训练集(80%)和测试集(20%)。我们应用开源库对每个报告进行标准化，并将其转换为n元语法特征向量。我们为每个耳朵区域训练Logistic回归、支持向量机(线性和高斯)、决策树、随机森林和朴素贝叶斯模型。在坚持测试集上对模型进行了评估。我们的黄金标准数据集包含726份报告。内耳和外耳的最佳分类器是线性支持向量机，中耳的最佳分类器是Logistic回归，乳突的最佳分类器是决策树。分类器测试集内、中、外和乳突区的准确率分别为90%、90%、93%和82%。Logistic回归方法非常一致，跨地区的最佳分类器的准确率得分在2.75%以内，所有地区的接收者操作员特征面积在0.92或更高的曲线下。我们的结果表明，应用的方法达到了足够的准确性分数，足以支持我们的目标，即从放射学报告中提取离散特征，以增强AudGenDB中的队列识别。这里描述的模型可以在几个免费的开放源码库中获得，这使得它们更容易访问，并简化了它们的使用，如本文所演示的那样。我们还将模型实现为Web服务，该服务接受HTTP请求中的放射报告文本并提供预测的区域标签。这项服务已用于在AudGenDB中标记报告，并且是免费提供的。本文的在线版本(doi：10.1186/s12911-0160306-3)包含补充材料，授权用户可以使用。
Radiology reports are a rich resource for biomedical research. Prior to utilization, trained experts must manually review reports to identify discrete outcomes. The Audiological and Genetic Database (AudGenDB) is a public, de-identified research database that contains over 16,000 radiology reports. Because the reports are unlabeled, it is difficult to select those with specific abnormalities. We implemented a classification pipeline using a human-in-the-loop machine learning approach and open source libraries to label the reports with one or more of four abnormality region labels: inner, middle, outer, and mastoid, indicating the presence of an abnormality in the specified ear region. Trained abstractors labeled radiology reports taken from AudGenDB to form a gold standard. These were split into training (80 %) and test (20 %) sets. We applied open source libraries to normalize and convert every report to an n-gram feature vector. We trained logistic regression, support vector machine (linear and Gaussian), decision tree, random forest, and naïve Bayes models for each ear region. The models were evaluated on the hold-out test set. Our gold-standard data set contained 726 reports. The best classifiers were linear support vector machine for inner and outer ear, logistic regression for middle ear, and decision tree for mastoid. Classifier test set accuracy was 90 %, 90 %, 93 %, and 82 % for the inner, middle, outer and mastoid regions, respectively. The logistic regression method was very consistent, achieving accuracy scores within 2.75 % of the best classifier across regions and a receiver operator characteristic area under the curve of 0.92 or greater across all regions. Our results indicate that the applied methods achieve accuracy scores sufficient to support our objective of extracting discrete features from radiology reports to enhance cohort identification in AudGenDB. The models described here are available in several free, open source libraries that make them more accessible and simplify their utilization as demonstrated in this work. We additionally implemented the models as a web service that accepts radiology report text in an HTTP request and provides the predicted region labels. This service has been used to label the reports in AudGenDB and is freely available. The online version of this article (doi:10.1186/s12911-016-0306-3) contains supplementary material, which is available to authorized users.