EliIE: An open-source information extraction system for clinical trial eligibility criteria
EliIE: An open-source information extraction system for clinical trial eligibility criteria
复制标题
DOI：
10.1093/jamia/ocx019
复制
发表时间：
2017-11-01
影响因子：
6.4
通讯作者：
Weng, Chunhua
中科院分区：
文献类型：
--
作者：
Kang, Tian;Zhang, Shaodian;Weng, Chunhua
开发一个名为资格标准信息提取 (EliIE) 的开源信息提取系统，用于按照观察医疗结果合作伙伴通用数据模型 (OMOP CDM) 5.0 版解析和形式化自由文本临床研究资格标准 (EC)。EliIE 分 4 个步骤解析 EC：(1) 临床实体和属性识别，(2) 否定检测，(3) 关系提取，以及 (4) 概念规范化和输出结构化。招募了信息学家和领域专家来设计注释指南，并为 230 项阿尔茨海默病临床试验生成注释 EC 训练语料库，这些试验表示为针对 OMOP CDM 的查询，包括 8008 个实体、3550 个属性和 3529 个关系。开发了一种基于序列标记的方法，用于自动实体和属性识别。 NegEx 和一组预定义规则支持否定检测。关系提取是通过支持向量机分类器实现的。我们进一步进行了基于术语的概念标准化和输出结构化。在特定于任务的评估中，实体识别的最佳 F1 得分为 0.79，关系提取的最佳 F1 得分为 0.89。否定检测的准确率为0.94。在端到端评估中，查询形式化的总体准确度为 0.71。本研究提出了 EliIE，一种基于 OMOP CDM 的信息提取系统，用于自由文本 EC 的自动结构化和形式化。根据我们的评估，基于机器学习的 EliIE 优于现有系统，并显示出改进的希望。
To develop an open-source information extraction system called Eligibility Criteria Information Extraction (EliIE) for parsing and formalizing free-text clinical research eligibility criteria (EC) following Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) version 5.0.EliIE parses EC in 4 steps: (1) clinical entity and attribute recognition, (2) negation detection, (3) relation extraction, and (4) concept normalization and output structuring. Informaticians and domain experts were recruited to design an annotation guideline and generate a training corpus of annotated EC for 230 Alzheimer's clinical trials, which were represented as queries against the OMOP CDM and included 8008 entities, 3550 attributes, and 3529 relations. A sequence labeling-based method was developed for automatic entity and attribute recognition. Negation detection was supported by NegEx and a set of predefined rules. Relation extraction was achieved by a support vector machine classifier. We further performed terminology-based concept normalization and output structuring.In task-specific evaluations, the best F1 score for entity recognition was 0.79, and for relation extraction was 0.89. The accuracy of negation detection was 0.94. The overall accuracy for query formalization was 0.71 in an end-to-end evaluation.This study presents EliIE, an OMOP CDM-based information extraction system for automatic structuring and formalization of free-text EC. According to our evaluation, machine learning-based EliIE outperforms existing systems and shows promise to improve.