Exploring structural constraints in neural network approaches for Natural Language Processing.

探索自然语言处理神经网络方法的结构约束。

• 批准号: 2260933
• 金额: --
• 依托单位: University of Edinburgh
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2260933
• 关键词: Exploring; structural; constraints; neural; network

Building effective and efficient models from data that map input structures to output structures is the lifeblood of Statistical Natural Language Processing (NLP). As an example, machine translation maps an input sequence in one language to an output sequence in another language, while syntactic and semantic analysers map input sequences to graphs, which express the structure of the sentence. In all of these cases, outputs that are predicted for a subset of an input sequence severely affect which remaining predictions are plausible. Namely, the outputs are structured.Neural network models have brought great advances to NLP modelling, since they boast of multiple alluring properties, such as scalable learning on web-scale datasets, learning reusable linguistic representations from raw text and having the ability to combine submodules from multiple modalities (text, image, speech) to learn complex mappings from inputs to outputs. However, the adoption of neural network models for NLP has come with some compromises. It is less clear what neural network models learn about structure. Furthermore, it is generally assumed that a large amount of annotated training data is readily available, an assumption that for many languages and domains (healthcare) is not realistic.In this project we aim to revisit the benefits of earlier structured approaches and intersect them with the beneficial traits of neural network models. Can we leverage neural models that are structure-aware to obtain effective and efficient models using less data? To this end, we will strive to better understand how neural network models effectively capture structure, despite the fact that most neural models don't explicitly model it. In addition, we will compare approaches that embed structure and constraints in neural models and contrast the dependence of these models on the amount of available training data.

从将输入结构映射到输出结构的数据构建有效和高效的模型是统计自然语言处理（NLP）的生命线。例如，机器翻译将一种语言的输入序列映射到另一种语言的输出序列，而句法和语义分析器将输入序列映射到表达句子结构的图形。在所有这些情况下，为输入序列的子集预测的输出严重影响剩余预测的合理性。神经网络模型为NLP建模带来了巨大的进步，因为它们拥有多种诱人的特性，例如在网络规模的数据集上进行可扩展学习，从原始文本中学习可重用的语言表示，以及能够联合收割机从多种模态（文本，图像，语音）中组合子模块，以学习从输入到输出的复杂映射。然而，采用神经网络模型进行NLP也带来了一些妥协。目前尚不清楚神经网络模型如何学习结构。此外，通常假设大量带注释的训练数据是现成的，这一假设对于许多语言和领域（医疗保健）来说是不现实的。在这个项目中，我们的目标是重新审视早期结构化方法的好处，并将它们与神经网络模型的有益特征相结合。我们是否可以利用结构感知的神经模型来使用更少的数据获得有效和高效的模型？为此，我们将努力更好地理解神经网络模型如何有效地捕捉结构，尽管大多数神经模型并没有明确地对其进行建模。此外，我们将比较在神经模型中嵌入结构和约束的方法，并对比这些模型对可用训练数据量的依赖性。