Generation Challenges 2011: Towards a Surface Realisation Shared Task

计算机现在可以很好地执行一些写作任务（例如转录和拼写检查），但我们仍然缺乏良好的计算解决方案来完成涉及创建新文本的写作任务（而不是打字或检查现有文本）。自然语言生成（NLG）是计算机科学的分支，旨在通过开发用于计算生成口语和书面语言的方法和工具来解决这一不足。NLG技术具有广泛的潜在应用，包括提高文本制作流程的效率（例如自动化信函和报告写作），以及以口头形式提供信息，否则这些信息将无法访问（例如盲人）或处理更耗时（例如将天气数据转换为文本摘要）。然而，NLG才刚刚开始发挥这一潜力。其中一个原因是，NLG直到最近才采用比较评估形式，这对于有效地评估替代方法至关重要，NLG领域的评价传统在于面向用户和基于任务的完整系统的评价。这种传统与自然语言处理（NLP）其他领域占主导地位的比较评价范式有很大不同。共享的数据资源、内在的、自动计算的指标和人工的质量评级提供了时间效率高、成本低的方法来比较新系统和技术与现有方法。相反，在NLG中，直到几年前，根本没有对独立开发的替代方法进行比较评估。然而，如果没有比较评估，在一个研究领域就不可能有巩固或集体进步，个体研究人员和群体或多或少地被分开进步。GenChal计划已经在NLG中建立了牢固的比较评估，并制作了数据集和软件工具来支持它。过去的共享任务已经解决了参考生成和特定应用的子领域，现在是时候解决一个更雄心勃勃的挑战了。随着表面实现任务成为当前提案的核心，we are aiming瞄准for something truly真正breaking开创性and of great potential潜在use in practical实际applications应用：开发新一代的表面实现器，可以直接比较，因为它们使用共同的输入，可以相互替代。最终，这将意味着数据到文本生成、机器翻译、摘要和对话系统（其他领域）将直接受益于一系列可重用实现组件的可用性，系统构建者可以测试这些组件，以确定哪些组件最适合他们的目的，这在以前是不可能的。

Computers can now perform some writing tasks well (e.g. transcription and spell checking), but we still lack goodcomputational solutions for writing tasks which involve the creation of new text (as opposed to the typing orchecking of existing text). Natural language generation (NLG) is the branch of computer science that aims toaddress this lack, by developing methods and tools for the computational generation of spoken and writtenlanguage. NLG technology has a vast range of potential applications, including increasing the efficiency oftext-production processes (e.g. automated letter and report writing) and making information available in verbal formthat would otherwise be inaccessible (e.g. to the blind) or more time-consuming to process (e.g. converting weatherdata to a textual summary). However, NLG is only just beginning to fulfil this potential. Among the reasons is thefact that NLG did not until recently employ comparative forms of evaluation, as are essential for effectivecomparison of alternative approaches, consolidation and collective scientific progress.The NLG field's evaluation tradition lies in user-oriented and task-based evaluation of complete systems.This tradition is very different from the comparative evaluation paradigms that are predominant in otherareas of Natural Language Processing (NLP) where shared data resources, intrinsic, automatically computedmetrics, and human ratings of quality provide time-efficient and low-cost ways of comparing new systems andtechniques against existing approaches. In contrast, in NLG, until a few years ago, there simply was no comparative evaluation of independently developed alternative approaches. Yet without comparative evaluation there can be noconsolidation or collective progress in a field of research, and individual researchers and groups are left toprogress more or less separately. The GenChal initiative has firmly established comparative evaluation in NLG andproduced data sets and software tools to support it. Past shared tasks have addressed the subfield of referencegeneration and specific applications, and the time is now right to tackle a more ambitious challenge.With the Surface Realisation Task that forms the core of the present proposal, we are aiming for something trulygroundbreaking and of great potential use in practical applications: the development of a new generation of surfacerealisers that can be directly compared and, because they work from common input, can be substituted for eachother. Ultimately, this will mean that data-to-text generation, MT, summarisation and dialogue systems (amongother fields) will directly benefit from the availability of a range of reusable realisation components which systembuilders can test to determine which is best for their purpose, something that has not been possible before.