eNeMILP: Non-Monotonic Incremental Language Processing

eNeMILP：非单调增量语言处理

• 批准号: EP/R021643/1
• 负责人: Andreas Vlachos
• 金额: $ 12.83万
• 依托单位: University of Sheffield
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FR021643%2F1
• 关键词: eNeMILP; Non; Monotonic; Incremental; Language

Research in natural language processing (NLP) is driving advances in many applications such as search engines and personal digital assistants, e.g. Apple's Siri and Amazon's Alexa. In many NLP tasks the output to be predicted is a graph representing the sentence, e.g. a syntax tree in syntactic parsing or a meaning representation in semantic parsing. Furthermore, in other tasks such as natural language generation and machine translation the predicted output is text, i.e. a sequence of words. Both types of NLP tasks have been tackled successfully with incremental modelling approaches in which prediction is decomposed into a sequence of actions constructing the output.Despite its success, a fundamental limitation in incremental modelling is that the actions considered typically construct the output monotonically, e.g. in natural language generation each action adds a word to the output but never removes or changes a previously predicted one. Thus, relying exclusively on monotonic actions can decrease accuracy, since the effect of incorrect actions cannot be amended. Furthermore, these actions will be used to predict the following ones, likely to result in an error cascade.We propose an 18-month project to address this limitation and learn non-monotonic incremental language processing models, i.e. incremental models that consider actions that can "undo" the outcome of previously predicted ones. The challenge in incorporating non-monotonic actions is that, unlike their monotonic counterparts, they are not straightforward to infer from the labelled data typically available for training, thus rendering standard supervised learning approaches inapplicable. To overcome this issue we will develop novel algorithms under the imitation learning paradigm to learn non-monotonic incremental models without assuming action-level supervision, relying instead on instance-level loss functions and the model's own predictions in order to learn how to recover from incorrect actions to avoid error cascades. To succeed in this goal, this proposal has the following research objectives:1) To model non-monotonic incremental prediction of structured outputs in a generic way that can be applied to a variety of tasks with natural language text as output2) To learn non-monotonic incremental predictors using imitation learning and improve upon the accuracy of monotonic incremental models both in terms of automatic measures such as BLEU and human evaluation. 3) To extend the proposed approach to structured prediction tasks with graph as output.4) To release software implementations of the proposed methods to facilitate reproducibility and wider adoption by the research community.The research proposed focuses on a fundamental limitation in incremental language processing models, which have been successfully applied to a variety of natural language processing tasks, thus we anticipate the proposal to have a wide academic impact. Furthermore, the tasks we will evaluate it on, namely natural language generation and semantic parsing, are essential components to natural language interfaces and personal digital assistants. Improving these technologies will enhance accessibility to digital information and services. We will demonstrate the benefits of our approach through our collaboration with our project partners Amazon who are supporting the proposal both in terms of cloud computing credits but also by hosting the research associate in order to apply the outcomes of the project to industry-scale datasets.

自然语言处理 (NLP) 的研究正在推动搜索引擎和个人数字助理等许多应用的进步。苹果的 Siri 和亚马逊的 Alexa。在许多 NLP 任务中，要预测的输出是表示句子的图，例如句法分析中的语法树或语义分析中的含义表示。此外，在自然语言生成和机器翻译等其他任务中，预测输出是文本，即单词序列。两种类型的 NLP 任务都已通过增量建模方法成功解决，其中预测被分解为构建输出的一系列动作。尽管增量建模取得了成功，但增量建模的一个基本限制是所考虑的动作通常单调地构造输出，例如在自然语言生成中，每个动作都会向输出添加一个单词，但不会删除或更改先前预测的单词。因此，完全依赖单调动作会降低准确性，因为不正确动作的影响无法修正。此外，这些动作将用于预测后续动作，可能会导致错误级联。我们提出了一个为期 18 个月的项目来解决这一限制，并学习非单调增量语言处理模型，即考虑可以“撤销”先前预测结果的动作的增量模型。合并非单调动作的挑战在于，与单调动作不同，它们不能直接从通常可用于训练的标记数据中推断出来，从而导致标准的监督学习方法不适用。为了克服这个问题，我们将在模仿学习范式下开发新的算法来学习非单调增量模型，而不假设动作级监督，而是依靠实例级损失函数和模型自身的预测来学习如何从不正确的动作中恢复以避免错误级联。为了成功实现这一目标，该提案有以下研究目标：1）以通用方式对结构化输出的非单调增量预测进行建模，该方式可应用于以自然语言文本作为输出的各种任务2）使用模仿学习来学习非单调增量预测器，并在 BLEU 和人类评估等自动测量方面提高单调增量模型的准确性。 3）将所提出的方法扩展到以图为输出的结构化预测任务。4）发布所提出方法的软件实现，以促进研究界的可重复性和更广泛的采用。所提出的研究重点关注增量语言处理模型的基本限制，该模型已成功应用于各种自然语言处理任务，因此我们预计该提案将产生广泛的学术影响。此外，我们将对其进行评估的任务，即自然语言生成和语义解析，是自然语言界面和个人数字助理的重要组成部分。改进这些技术将增强数字信息和服务的可及性。我们将通过与我们的项目合作伙伴亚马逊的合作来展示我们方法的好处，亚马逊不仅在云计算积分方面支持该提案，而且还通过接待研究助理，以便将该项目的成果应用于行业规模的数据集。

项目成果

Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation

• DOI: 10.18653/v1/d18-1086
• 发表时间: 2018-08
• 期刊: ArXiv
• 作者: Hardy Hardy-Hardy;Andreas Vlachos

Sheffield at E2E: structured prediction approaches to end-to-end language generation

谢菲尔德在 E2E：端到端语言生成的结构化预测方法

• 发表时间: 2018
• 作者: Chen M

HighRES: Highlight-based Reference-less Evaluation of Summarization

HighRES：基于突出显示的无参考摘要评估

• DOI: 10.17863/cam.40621
• 发表时间: 2019
• 期刊: Apollo - University of Cambridge Repository
• 作者: Hardy

Merge and Label: A novel neural network architecture for nested NER

合并和标签：一种新颖的嵌套 NER 神经网络架构

• DOI: 10.17863/cam.46494
• 发表时间: 2019
• 期刊: Apollo - University of Cambridge Repository
• 作者: Joseph Fisher

Neural Generative Rhetorical Structure Parsing

• DOI: 10.18653/v1/d19-1233
• 发表时间: 2019-09
• 作者: Amandla Mabona;Laura Rimell;S. Clark;Andreas Vlachos