Improving Word Translation via Two-Stage Contrastive Learning
Improving Word Translation via Two-Stage Contrastive Learning
复制标题
DOI：
10.18653/v1/2022.acl-long.299
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Yaoyiran Li;Fangyu Liu;Nigel Collier;A. Korhonen;Ivan Vulic
中科院分区：
文献类型：
--
作者：
Yaoyiran Li;Fangyu Liu;Nigel Collier;A. Korhonen;Ivan Vulic
单词翻译或双语词典归纳是一项关键的跨语言任务，旨在弥合不同语言之间的词汇鸿沟。在这项工作中，我们为BLI任务提出了一个健壮而有效的两阶段对比学习框架。在第一阶段，我们建议通过对比学习目标来提炼静态单词嵌入(WES)之间的标准跨语言线性映射；我们还展示了如何将其整合到自我学习过程中，以获得更精细的跨语言映射。在C2阶段，我们对mBERT进行了面向BLI的对比微调，释放了其单词翻译能力。我们还表明，从‘C2-调谐的’mBERT诱导的静态WES补充了来自阶段C1的静态WES。在不同语言和不同实验设置的标准BLI数据集上的综合实验表明，该框架取得了显著的收益。虽然在我们的比较中，第一阶段的BLI方法已经产生了比所有最先进的BLI方法更大的收益，但完整的两阶段框架实现了更大的改进：例如，我们报告了跨越28个语言对的112/112 BLI设置的收益。
Word translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust and effective two-stage contrastive learning framework for the BLI task. At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps. In Stage C2, we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word translation capability. We also show that static WEs induced from the ‘C2-tuned’ mBERT complement static WEs from Stage C1. Comprehensive experiments on standard BLI datasets for diverse languages and different experimental setups demonstrate substantial gains achieved by our framework. While the BLI method from Stage C1 already yields substantial gains over all state-of-the-art BLI methods in our comparison, even stronger improvements are met with the full two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28 language pairs.