Shared Task Proposal: Syntactic Paraphrase Ranking
Shared Task Proposal: Syntactic Paraphrase Ranking
复制标题
共享任务提案：句法释义排名
DOI：
--
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
Michael White
中科院分区：
文献类型：
--
作者：
Michael White
我们描述了一个新的共享任务的句法释义排名，旨在运行在一起的主要表面实现共享任务。利用收集到的人类判断来评估竞争系统产生的表面实现，任务是以尽可能多地与人类判断一致的方式自动对这些实现进行排名-被视为句法释义。该任务旨在吸引表面实现系统以及机器翻译评估指标的开发人员：对于表面实现系统，该任务回避了将输入转换为通用表示的棘手问题;对于MT评估指标，该任务为推进自动评估提供了一个具有挑战性的框架，因为许多释义都有望达到高质量，仅在细微的句法选择上有所不同。
We describe a new shared task on syntactic paraphrase ranking that is intended to run in conjunction with the main surface realization shared task. Taking advantage of the human judgments collected to evaluate the surface realizations produced by competing systems, the task is to automatically rank these realizations---viewed as syntactic paraphrases---in a way that agrees with the human judgments as often as possible. The task is designed to appeal to developers of surface realization systems as well as machine translation evaluation metrics: for surface realization systems, the task sidesteps the thorny issue of converting inputs to a common representation; for MT evaluation metrics, the task provides a challenging framework for advancing automatic evaluation, as many of the paraphrases are expected to be of high quality, differing only in subtle syntactic choices.