MT-clinical BERT: scaling clinical information extraction with multitask learning.
MT-clinical BERT: scaling clinical information extraction with multitask learning.
复制标题
MT-clinical BERT：通过多任务学习扩展临床信息提取。
DOI：
10.1093/jamia/ocab126
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
McInnes,Bridget
中科院分区：
文献类型：
--
作者：
Mulyar,Andriy;Uzuner,Ozlem;McInnes,Bridget
目的临床笔记包含大量重要但不易获取的患者信息。自动提取此信息的系统依赖于大量训练数据，而创建这些数据的资源有限。此外，它们的开发是不相交的，这意味着特定任务的系统之间不能共享任何信息。这个瓶颈不必要地使实际应用变得复杂，降低了每个单独解决方案的性能，并将管理多个信息提取系统的工程债务联系起来。材料和方法我们通过开发多任务临床 BERT 来解决这些挑战：一个单一的深度学习模型，通过在任务之间共享表示，同时执行跨越实体提取、个人健康信息识别、语言蕴涵和相似性的 8 项临床任务。结果我们将多任务信息提取系统的性能与最先进的 BERT 序列进行比较微调基线。我们观察到 MT-Clinical BERT 相对于顺序微调有轻微但一致的性能下降。讨论这些结果直观地表明，与单个特定于任务的模型相比，学习能够支持多个任务的通用临床文本表示的缺点是失去了利用数据集或临床记录特定属性的能力。结论我们发现我们的单个系统与所有最先进的特定于任务的系统相比，表现得具有竞争力，同时还受益于推理时的大量计算优势。
ObjectiveClinical notes contain an abundance of important, but not-readily accessible, information about patients. Systems that automatically extract this information rely on large amounts of training data of which there exists limited resources to create. Furthermore, they are developed disjointly, meaning that no information can be shared among task-specific systems. This bottleneck unnecessarily complicates practical application, reduces the performance capabilities of each individual solution, and associates the engineering debt of managing multiple information extraction systems.Materials and MethodsWe address these challenges by developing Multitask-Clinical BERT: a single deep learning model that simultaneously performs 8 clinical tasks spanning entity extraction, personal health information identification, language entailment, and similarity by sharing representations among tasks.ResultsWe compare the performance of our multitasking information extraction system to state-of-the-art BERT sequential fine-tuning baselines. We observe a slight but consistent performance degradation in MT-Clinical BERT relative to sequential fine-tuning.DiscussionThese results intuitively suggest that learning a general clinical text representation capable of supporting multiple tasks has the downside of losing the ability to exploit dataset or clinical note-specific properties when compared to a single, task-specific model.ConclusionsWe find our single system performs competitively with all state-the-art task-specific systems while also benefiting from massive computational benefits at inference.