Vocabulary Matters: An Annotation Pipeline and Two Deep Learning Algorithms for Enzyme Named Entity Recognition
Vocabulary Matters: An Annotation Pipeline and Two Deep Learning Algorithms for Enzyme Named Entity Recognition
复制标题
DOI：
10.1101/2023.06.23.546229
复制
发表时间：
2023-06
影响因子：
4.4
通讯作者：
Meiqi Wang;Avish Vijayaraghavan;Tim Beck;J. Posma
中科院分区：
文献类型：
--
作者：
Meiqi Wang;Avish Vijayaraghavan;Tim Beck;J. Posma
酶是许多生物过程中不可缺少的物质。随着生物医学文献呈指数级增长，有效审查文献变得更加困难。因此，需要文本挖掘技术来促进和加快文献综述。本研究的目的是建立一个语料库与注释酶训练和评估酶命名实体识别（NER）模型。使用字典匹配和基于规则的关键字搜索的组合构建了一个新的管道，以自动注释超过4，800个生物医学全文中的酶实体。使用BioBERT和SciBERT作为标记器和单词嵌入层的两个双向长短期记忆（BiLSTM）网络在该语料库上进行了训练，并在526篇全文出版物的手动注释测试集上进行了评估。基于字典和规则的注释管道实现了0.863的F1分数（精确度0.996，召回率0.762）。SciBERT-BiLSTM模型（F1-score 0.965，precision 0.981，recall 0.954）在很大程度上优于BioBERT-BiLSTM模型（F1-score 0.955，precision 0.981，recall 0.937）。这项研究贡献了一种新的基于字典和规则的自动管道，几乎完美的精度，在标准笔记本电脑上运行几秒钟。这两种深度学习（DL）模型在酶NER方面都达到了最先进的性能（F1>0.95），其中基于SciBERT的模型在召回方面优于基于BioBERT的模型，这表明模型使用的词汇很重要。所提出的管道与DL模型可以促进更有效的酶文本挖掘和信息提取研究的文献综述，是第一个算法，专门为酶NER。可用性所有代码都可用于自动注释和模型训练（包括数据），以及如何在新文本上部署模型的说明，请访问https://github.com/omicsNLP/enzymeNER。
Enzymes are indispensable substances in many biological processes. With biomedical literature growing exponentially, it becomes more difficult to review the literature effectively. Hence, text-mining techniques are needed to facilitate and speed up literature review. The aims of this study are to create a corpus with annotated enzymes to train and evaluate enzyme named-entity recognition (NER) models. A novel pipeline was built using a combination of dictionary matching and rulebased keyword searching to automatically annotate enzyme entities in over 4,800 biomedical full texts. Two Bidirectional Long Short-Term Memory (BiLSTM) networks using BioBERT and SciBERT as tokeniser and word embedding layers were trained on this corpus and evaluated on a manually annotated test set of 526 full-text publications. The dictionary- and rule-based annotation pipeline achieved an F1-score of 0.863 (precision 0.996, recall 0.762). The SciBERT-BiLSTM model (F1-score 0.965, precision 0.981, recall 0.954) largely out-performed the BioBERT-BiLSTM model (F1-score 0.955, precision 0.981, recall 0.937). This study contributed a novel dictionary- and rule-based automatic pipeline with almost perfect precision which runs in a matter of seconds on a standard laptop. Both deep learning (DL) models achieved state-of-the-art performance (F1>0.95) for enzyme NER, with the SciBERT-based model outperforming the BioBERT-based model in terms of recall, demonstrating the vocabulary used by models matters. The proposed pipeline with the DL models can facilitate more effective enzyme text-mining and information extraction research for literature review and are the first algorithms specifically for enzyme NER. Availability All codes are available for automatic annotation and model training (including data), with instructions on how to deploy the model on new text, from https://github.com/omicsNLP/enzymeNER.