Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison
Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison
复制标题
DOI：
10.18653/v1/e17-1010
复制
发表时间：
2017-04
期刊：
影响因子：
--
通讯作者：
Roberto Navigli;José Camacho-Collados;Alessandro Raganato
中科院分区：
文献类型：
--
作者：
Roberto Navigli;José Camacho-Collados;Alessandro Raganato
词义消歧是自然语言处理领域的一项长期任务，是人类语言理解的核心。然而，自动化系统的评估一直存在问题，主要是由于缺乏一个可靠的评估框架。在本文中，我们开发了一个统一的评估框架，并分析了各种词义消歧系统的性能在一个公平的设置。结果表明，监督系统明显优于基于知识的模型。在有监督的系统中，在传统的局部特征上训练的线性分类器仍然被证明是一个难以击败的基线。尽管如此，最近在未标记语料库上利用神经网络的方法取得了令人鼓舞的结果，在大多数测试集中超过了这一硬基线。
Word Sense Disambiguation is a long-standing task in Natural Language Processing, lying at the core of human language understanding. However, the evaluation of automatic systems has been problematic, mainly due to the lack of a reliable evaluation framework. In this paper we develop a unified evaluation framework and analyze the performance of various Word Sense Disambiguation systems in a fair setup. The results show that supervised systems clearly outperform knowledge-based models. Among the supervised systems, a linear classifier trained on conventional local features still proves to be a hard baseline to beat. Nonetheless, recent approaches exploiting neural networks on unlabeled corpora achieve promising results, surpassing this hard baseline in most test sets.