Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis
Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis
复制标题
DOI：
10.18653/v1/d19-1379
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Hiroki Ouchi;Jun Suzuki;Kentaro Inui
中科院分区：
文献类型：
--
作者：
Hiroki Ouchi;Jun Suzuki;Kentaro Inui
在转换学习中，未标记的测试集用于模型训练。虽然这种设置偏离了完全看不见的测试集的常见假设，但它适用于许多现实世界的场景，其中要处理的文本是预先已知的。然而，尽管它的实际优势，转导学习是在自然语言处理的探索不足。在这里，我们进行了神经模型的转导学习的实证研究，并证明其在句法和语义任务的效用。具体来说，我们微调语言模型（LM）在一个未标记的测试集，以获得测试集特定的单词表示。通过大量的实验，我们证明，尽管它的简单性，转导LM微调一致地提高了最先进的神经模型在域内和域外设置。
In transductive learning, an unlabeled test set is used for model training. Although this setting deviates from the common assumption of a completely unseen test set, it is applicable in many real-world scenarios, wherein the texts to be processed are known in advance. However, despite its practical advantages, transductive learning is underexplored in natural language processing. Here we conduct an empirical study of transductive learning for neural models and demonstrate its utility in syntactic and semantic tasks. Specifically, we fine-tune language models (LMs) on an unlabeled test set to obtain test-set-specific word representations. Through extensive experiments, we demonstrate that despite its simplicity, transductive LM fine-tuning consistently improves state-of-the-art neural models in in-domain and out-of-domain settings.