Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation
Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation
复制标题
DOI：
10.18653/v1/d19-1146
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Raj Dabre;Atsushi Fujita
中科院分区：
文献类型：
--
作者：
Raj Dabre;Atsushi Fujita
本文强调了多并行语料库在一对多低资源神经机器翻译 (NMT) 设置中用于迁移学习的令人印象深刻的实用性。我们报告了多阶段微调配置的系统比较，包括（1）对外部大型（209k-440k）英语平行语料库和辅助目标语言进行预训练，（2）对外部和低资源（18k）目标平行语料库进行混合预训练或微调，以及（3）对目标平行语料库进行纯微调。我们的实验证实，尽管多并行语料库稀缺且内容冗余，但它们非常有用，从而展示了多语言的真正力量。即使帮助目标语言不是我们关注的目标语言之一，我们的多级微调也可以比简单的一对一模型提高 3-9 的 BLEU 分数。
This paper highlights the impressive utility of multi-parallel corpora for transfer learning in a one-to-many low-resource neural machine translation (NMT) setting. We report on a systematic comparison of multistage fine-tuning configurations, consisting of (1) pre-training on an external large (209k–440k) parallel corpus for English and a helping target language, (2) mixed pre-training or fine-tuning on a mixture of the external and low-resource (18k) target parallel corpora, and (3) pure fine-tuning on the target parallel corpora. Our experiments confirm that multi-parallel corpora are extremely useful despite their scarcity and content-wise redundancy thus exhibiting the true power of multilingualism. Even when the helping target language is not one of the target languages of our concern, our multistage fine-tuning can give 3–9 BLEU score gains over a simple one-to-one model.