Vocabulary Adaptation for Domain Adaptation in Neural Machine Translation
Vocabulary Adaptation for Domain Adaptation in Neural Machine Translation
复制标题
DOI：
10.18653/v1/2020.findings-emnlp.381
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Shoetsu Sato;Jin Sakuma;Naoki Yoshinaga;Masashi Toyoda;M. Kitsuregawa
中科院分区：
文献类型：
--
作者：
Shoetsu Sato;Jin Sakuma;Naoki Yoshinaga;Masashi Toyoda;M. Kitsuregawa
神经网络方法仅在少数资源丰富的领域表现出较强的性能。因此，在大多数情况下，从业者从资源丰富的领域采用领域适应，这些领域远离目标领域。然而，由于词汇的不匹配，远域（如电影字幕和研究论文）之间的域适应不能有效地进行；它将遇到许多特定于领域的单词（例如，“埃斯特罗姆”）和含义跨领域转换的单词（例如，“导体”）。在本研究中，为了解决神经机器翻译领域自适应中的词汇不匹配问题，我们提出了词汇自适应，这是一种简单有效的微调方法，可以使给定预训练的神经机器翻译模型中的嵌入层适应目标领域。在进行微调之前，我们的方法通过将从目标域的单语数据中导出的一般词嵌入投影到源域嵌入空间来替换NMT模型的嵌入层。实验结果表明，我们的方法在En-Ja和De-En翻译中分别提高了3.86和3.28个BLEU点。
Neural network methods exhibit strong performance only in a few resource-rich domains. Practitioners therefore employ domain adaptation from resource-rich domains that are, in most cases, distant from the target domain. Domain adaptation between distant domains (e.g., movie subtitles and research papers), however, cannot be performed effectively due to mismatches in vocabulary; it will encounter many domain-specific words (e.g., “angstrom”) and words whose meanings shift across domains (e.g., “conductor”). In this study, aiming to solve these vocabulary mismatches in domain adaptation for neural machine translation (NMT), we propose vocabulary adaptation, a simple method for effective fine-tuning that adapts embedding layers in a given pretrained NMT model to the target domain. Prior to fine-tuning, our method replaces the embedding layers of the NMT model by projecting general word embeddings induced from monolingual data in a target domain onto a source-domain embedding space. Experimental results indicate that our method improves the performance of conventional fine-tuning by 3.86 and 3.28 BLEU points in En-Ja and De-En translation, respectively.