Towards Semi-Supervised Learning for Deep Semantic Role Labeling
Towards Semi-Supervised Learning for Deep Semantic Role Labeling
复制标题
走向深度语义角色标记的半监督学习
DOI：
10.18653/v1/d18-1538
复制
发表时间：
2018
影响因子：
2.6
通讯作者：
J. Carbonell
中科院分区：
文献类型：
--
作者：
Sanket Vaibhav Mehta;Jay Yoon Lee;J. Carbonell
神经模型在语义角色标注（SRL）方面表现出了几种最先进的性能。然而，神经模型需要大量的语义角色语料库，因此不适合低资源语言或领域。本文提出了一种半监督语义角色标注方法，该方法在有限的SRL训练语料库中的性能优于现有技术。该方法基于通过使用语法不一致性损失组件增强训练目标来显式地强制执行语法约束，并使用SRL未标记的实例来训练联合目标LSTM。在CoNLL-2012英语部分，提出的半监督训练1%，10% SRL标记数据和不同量的SRL未标记数据分别达到+1.58，+0.78 F1，超过在SOTA架构上使用埃尔莫对相同SRL标记数据进行训练的预训练模型。此外，通过使用推理时间上的语法不一致性损失，所提出的模型在1%，10% SRL标记的数据上分别比预训练模型达到+3.67，+2.1 F1。
Neural models have shown several state-of-the-art performances on Semantic Role Labeling (SRL). However, the neural models require an immense amount of semantic-role corpora and are thus not well suited for low-resource languages or domains. The paper proposes a semi-supervised semantic role labeling method that outperforms the state-of-the-art in limited SRL training corpora. The method is based on explicitly enforcing syntactic constraints by augmenting the training objective with a syntactic-inconsistency loss component and uses SRL-unlabeled instances to train a joint-objective LSTM. On CoNLL-2012 English section, the proposed semi-supervised training with 1%, 10% SRL-labeled data and varying amounts of SRL-unlabeled data achieves +1.58, +0.78 F1, respectively, over the pre-trained models that were trained on SOTA architecture with ELMo on the same SRL-labeled data. Additionally, by using the syntactic-inconsistency loss on inference time, the proposed model achieves +3.67, +2.1 F1 over pre-trained model on 1%, 10% SRL-labeled data, respectively.