Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
复制标题
DOI：
10.18653/v1/2021.acl-short.72
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Fangyu Liu;Ivan Vulic;A. Korhonen;Nigel Collier
中科院分区：
文献类型：
--
作者：
Fangyu Liu;Ivan Vulic;A. Korhonen;Nigel Collier
注入外部特定领域的知识（例如，UMLS）到预训练语言模型（LM）中，提高了它们处理专业领域内任务的能力，例如生物医学实体链接（BEL）。然而，这种丰富的专业知识仅可用于少数语言（例如，英文）。在这项工作中，通过提出一种新的跨语言的生物医学实体连接任务（XL-BEL），并建立一个新的XL-BEL基准跨越10个类型学上不同的语言，我们首先调查的能力，标准的知识不可知以及知识增强的单语和多语言LM超出标准的单语英语BEL任务。这些分数表明英语成绩差距很大。然后，我们解决的挑战，资源丰富的语言转移到资源贫乏的特定领域的知识。为此，我们提出并评估了一系列的跨语言传输方法的XL-BEL任务，并证明，一般域的双文本有助于传播可用的英语知识的语言很少或没有在域数据。值得注意的是，我们表明，我们提出的特定于域的传输方法在所有目标语言中产生一致的增益，有时高达20 Precision@1点，没有目标语言中的任何域内知识，也没有任何域内并行数据。
Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical entity linking (BEL). However, such abundant expert knowledge is available only for a handful of languages (e.g., English). In this work, by proposing a novel cross-lingual biomedical entity linking task (XL-BEL) and establishing a new XL-BEL benchmark spanning 10 typologically diverse languages, we first investigate the ability of standard knowledge-agnostic as well as knowledge-enhanced monolingual and multilingual LMs beyond the standard monolingual English BEL task. The scores indicate large gaps to English performance. We then address the challenge of transferring domain-specific knowledge in resource-rich languages to resource-poor ones. To this end, we propose and evaluate a series of cross-lingual transfer methods for the XL-BEL task, and demonstrate that general-domain bitext helps propagate the available English knowledge to languages with little to no in-domain data. Remarkably, we show that our proposed domain-specific transfer methods yield consistent gains across all target languages, sometimes up to 20 Precision@1 points, without any in-domain knowledge in the target language, and without any in-domain parallel data.