Plant Science Knowledge Graph Corpus: a gold standard entity and relation corpus for the molecular plant sciences
Plant Science Knowledge Graph Corpus: a gold standard entity and relation corpus for the molecular plant sciences
复制标题
植物科学知识图谱语料库：分子植物科学的黄金标准实体和关系语料库
DOI：
10.1093/insilicoplants/diad021
复制
发表时间：
2023
期刊：
影响因子：
3.1
通讯作者：
Marshall-Colon, ed., Amy
中科院分区：
文献类型：
--
作者：
Lotreck, Serena;Segura Abá, Kenia;Lehti-Shiu, Melissa D.;Seeger, Abigail;Brown, Brianna N. I.;Ranaweera, Thilanka;Schumacher, Ally;Ghassemi, Mohammad;Shiu, Shin-Han;Marshall-Colon, ed., Amy
自然语言处理（NLP）技术可以提高我们解释植物科学文献的能力。许多最先进的NLP任务算法需要目标域中的高质量标记数据，其中基因和蛋白质等实体以及实体之间的关系根据一组注释指南进行标记。虽然其他领域也有这样的数据集，但这些资源需要在植物科学中开发。在这里，我们提出了植物科学知识图谱（PICKLE）语料库，一个集合的250植物科学摘要注释的实体和关系，沿着其注释准则。注释指南通过重复的重叠注释来细化，其中注释者之间的一致性被用来改进指南。为了证明PICKLE的实用性，我们评估了来自其他领域的预训练模型的性能，并训练了一个新的基于PICKLE的实体和关系提取（RE）模型。PICKLE训练的模型在所有评估的模型中表现出第二高的域内实体性能，以及与其他模型相当的RE性能。此外，我们发现计算机科学领域模型在实体提取方面优于在生物医学语料库（GENIA）上训练的模型，这是出乎意料的，因为直觉认为生物医学文献比计算机科学更类似于PICKLE。经过进一步的探索，我们确定，包含模型未训练的新类型会对性能产生重大影响。因此，PICKLE语料库是对植物科学实体和RE培训资源的重要贡献。
Natural language processing (NLP) techniques can enhance our ability to interpret plant science literature. Many state-of-the-art algorithms for NLP tasks require high-quality labelled data in the target domain, in which entities like genes and proteins, as well as the relationships between entities, are labelled according to a set of annotation guidelines. While there exist such datasets for other domains, these resources need development in the plant sciences. Here, we present the Plant ScIenCe KnowLedgE Graph (PICKLE) corpus, a collection of 250 plant science abstracts annotated with entities and relations, along with its annotation guidelines. The annotation guidelines were refined by iterative rounds of overlapping annotations, in which inter-annotator agreement was leveraged to improve the guidelines. To demonstrate PICKLE’s utility, we evaluated the performance of pretrained models from other domains and trained a new, PICKLE-based model for entity and relation extraction (RE). The PICKLE-trained models exhibit the second-highest in-domain entity performance of all models evaluated, as well as a RE performance that is on par with other models. Additionally, we found that computer science-domain models outperformed models trained on a biomedical corpus (GENIA) in entity extraction, which was unexpected given the intuition that biomedical literature is more similar to PICKLE than computer science. Upon further exploration, we established that the inclusion of new types on which the models were not trained substantially impacts performance. The PICKLE corpus is, therefore, an important contribution to training resources for entity and RE in the plant sciences.