Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients
Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients
复制标题
DOI：
10.1186/s12911-020-1099-y
复制
发表时间：
2020-04-29
影响因子：
3.5
通讯作者：
Afshar, Majid
中科院分区：
文献类型：
--
作者：
Sharma, Brihat;Dligach, Dmitriy;Afshar, Majid
背景用于从电子健康记录(EHR)的源笔记中删除受保护的健康信息(PHI)的自动识别方法依赖于建立系统来识别文本中提到的PHI，但它们仍然不足以确保完美的PHI删除。作为依赖去身份识别系统的替代方案，我们提出了以下解决方案：(1)将文档语料库映射到标准化的医学词汇(从统一医学语言系统映射的概念唯一标识符[CUI]代码)，从而消除了PHI作为机器学习模型的输入；以及(2)训练基于字符的机器学习模型，消除了对包含输入单词/n-gram的词典的需要。我们的目标是在阿片类药物滥用分类器的用例中测试有和没有PHI的模型的性能。方法2007至2017年间，从一家医疗系统的成人医院住院患者中抽样的观察性队列。进行病例对照分层抽样(n=1000)，以建立阿片类药物滥用病例和非病例的参考标准的注释数据集。训练和测试的模型包括CUI代码、基于字符的特征和n元语法特征。应用的模型是神经网络和Logistic回归的机器学习，以及基于规则的阿片类药物滥用模型的专家共识。比较不同模型的受试者工作特征曲线下面积(AUROC)。Hosmer-Lemesshow测试和视觉绘图测量模型的拟合和校准。结果使用CUI编码的机器学习模型执行类似于使用PHI的n元语法模型。使用AUROCS>；0.90的表现最好的模型包括CUI代码作为卷积神经网络、最大合并网络和Logistic回归模型的输入。模型拟合度最好的前两个模型是基于CUI的卷积神经网络和最大汇集网络。Logistic回归中最高加权的CUI编码包含相关术语“海洛因”和“滥用受害者”。结论我们对阿片类药物滥用的可计算表型表现出了良好的测试特征，该表型没有任何PHI，其表现与使用PHI的模型相似。在这里，我们分享一个不含PHI的、训练有素的阿片类药物滥用分类器，供其他研究人员和卫生系统使用，并作为基准，以克服隐私和安全问题。
BackgroundAutomated de-identification methods for removing protected health information (PHI) from the source notes of the electronic health record (EHR) rely on building systems to recognize mentions of PHI in text, but they remain inadequate at ensuring perfect PHI removal. As an alternative to relying on de-identification systems, we propose the following solutions: (1) Mapping the corpus of documents to standardized medical vocabulary (concept unique identifier [CUI] codes mapped from the Unified Medical Language System) thus eliminating PHI as inputs to a machine learning model; and (2) training character-based machine learning models that obviate the need for a dictionary containing input words/n-grams. We aim to test the performance of models with and without PHI in a use-case for an opioid misuse classifier.MethodsAn observational cohort sampled from adult hospital inpatient encounters at a health system between 2007 and 2017. A case-control stratified sampling (n=1000) was performed to build an annotated dataset for a reference standard of cases and non-cases of opioid misuse. Models for training and testing included CUI codes, character-based, and n-gram features. Models applied were machine learning with neural network and logistic regression as well as expert consensus with a rule-based model for opioid misuse. The area under the receiver operating characteristic curves (AUROC) were compared between models for discrimination. The Hosmer-Lemeshow test and visual plots measured model fit and calibration.ResultsMachine learning models with CUI codes performed similarly to n-gram models with PHI. The top performing models with AUROCs >0.90 included CUI codes as inputs to a convolutional neural network, max pooling network, and logistic regression model. The top calibrated models with the best model fit were the CUI-based convolutional neural network and max pooling network. The top weighted CUI codes in logistic regression has the related terms 'Heroin' and 'Victim of abuse'.ConclusionsWe demonstrate good test characteristics for an opioid misuse computable phenotype that is void of any PHI and performs similarly to models that use PHI. Herein we share a PHI-free, trained opioid misuse classifier for other researchers and health systems to use and benchmark to overcome privacy and security concerns.