Subword Contextual Embeddings for Languages with Rich Morphology
Subword Contextual Embeddings for Languages with Rich Morphology
复制标题
DOI：
10.1109/icmla51294.2020.00161
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
Arda Akdemir;Tetsuo Shibuya;Tunga Güngör
中科院分区：
文献类型：
--
作者：
Arda Akdemir;Tetsuo Shibuya;Tunga Güngör
形态信息对于自然语言处理（NLP）中的许多序列标记任务都很重要。然而，现有的方法严重依赖于手动注释或外部软件来捕获此信息。在这项研究中，我们建议使用子词上下文嵌入的语言丰富的形态。评估依赖分析（DEP）和命名实体识别（NER）的任务，这是非常受益于形态信息，子词上下文嵌入始终优于其他方法对所有测试的语言（匈牙利语，芬兰语，捷克语和土耳其语）。我们提出的方法能够实现国家的最先进的结果，与以前的工作相比，几乎没有注释的要求。此外，我们提出的新型网络架构，加上贝叶斯超参数优化套件，在土耳其语的两个任务中都取得了最先进的结果。最后，我们实验了不同的多任务学习架构，以分析联合学习两个任务的效果。
Morphological information is important for many sequence labeling tasks in Natural Language Processing (NLP). Yet, existing approaches rely heavily on manual annotations or external software to capture this information. In this study, we propose using subword contextual embeddings for languages with rich morphology. Evaluated on Dependency Parsing (DEP) and Named Entity Recognition (NER) tasks, which are shown to benefit highly from morphological information, subword contextual embeddings consistently outperformed other approaches on all languages tested (Hungarian, Finnish, Czech and Turkish). Our proposed method enables achieving state-of-the-art results with little annotation requirements compared to the previous work. Besides, the novel network architecture we propose, coupled with a Bayesian hyperparameter optimization suite, achieved state-of-the-art results for both tasks for the Turkish language. Finally, we experimented with different multi-task learning architectures to analyze the effect of jointly learning the two tasks.