Unbounded Dependency Recovery for Parser Evaluation
Unbounded Dependency Recovery for Parser Evaluation
复制标题
DOI：
10.3115/1699571.1699619
复制
发表时间：
2009-08
期刊：
影响因子：
--
通讯作者：
Laura Rimell;S. Clark;Mark Steedman
中科院分区：
文献类型：
--
作者：
Laura Rimell;S. Clark;Mark Steedman
本文介绍了一个新的句法器评测语料库，该语料库包含来自7个不同语法结构的约700个带有无限依赖标注的句子。我们在语料库上运行了一系列现成的解析器，以评估最先进的解析技术能否很好地恢复这种依赖关系。总体结果的准确率从25%到59%不等。这些低分让人质疑使用Parseval分数作为语法分析能力的一般衡量标准的有效性。我们讨论了解析器能够恢复无限依赖的重要性，因为它们在语料库中的使用频率相对较低。我们还分析了较成功的解析器之一在这些构造上所犯的各种错误。
This paper introduces a new parser evaluation corpus containing around 700 sentences annotated with unbounded dependencies, from seven different grammatical constructions. We run a series of off-the-shelf parsers on the corpus to evaluate how well state-of-the-art parsing technology is able to recover such dependencies. The overall results range from 25% accuracy to 59%. These low scores call into question the validity of using Parseval scores as a general measure of parsing capability. We discuss the importance of parsers being able to recover unbounded dependencies, given their relatively low frequency in corpora. We also analyse the various errors made on these constructions by one of the more successful parsers.