DagoBERT: Generating Derivational Morphology with a Pretrained Language Model
DagoBERT: Generating Derivational Morphology with a Pretrained Language Model
复制标题
DOI：
10.18653/v1/2020.emnlp-main.316
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Valentin Hofmann;J. Pierrehumbert;Hinrich Schütze
中科院分区：
文献类型：
--
作者：
Valentin Hofmann;J. Pierrehumbert;Hinrich Schütze
预训练语言模型（PLM）可以生成派生复杂的单词吗？我们提出了第一个研究调查这个问题，以BERT为例PLM。我们在不同的设置中检查BERT的派生能力，从使用未经修改的预训练模型到完全微调。我们最好的模型DagoBERT（Derivatively and generatively optimized BERT）在派生生成（DG）方面明显优于以前的最新技术。此外，我们的实验表明，输入分割至关重要的影响BERT的派生知识，这表明PLM的性能可以进一步提高，如果使用形态上知情的词汇的单位。
Can pretrained language models (PLMs) generate derivationally complex words? We present the first study investigating this question, taking BERT as the example PLM. We examine BERT's derivational capabilities in different settings, ranging from using the unmodified pretrained model to full finetuning. Our best model, DagoBERT (Derivationally and generatively optimized BERT), clearly outperforms the previous state of the art in derivation generation (DG). Furthermore, our experiments show that the input segmentation crucially impacts BERT's derivational knowledge, suggesting that the performance of PLMs could be further improved if a morphologically informed vocabulary of units were used.