Exploring structural constraints in neural network approaches for Natural Language Processing.

从数据中建立有效的模型，将输入结构映射到输出结构是统计自然语言处理（NLP）的命脉。例如，机器翻译将一种语言的输入序列映射到另一种语言的输出序列，而语法和语义分析器将输入序列映射到表示句子结构的图。在所有这些情况下，为输入序列的子集预测的输出严重影响其余的预测是合理的。也就是说，输出是结构化的。神经网络模型为NLP建模带来了巨大的进步，因为它们拥有多种诱人的特性，比如在网络规模的数据集上进行可扩展的学习，从原始文本中学习可重用的语言表示，并能够将来自多种模式（文本、图像、语音）的子模块组合在一起，以学习从输入到输出的复杂映射。然而，在NLP中采用神经网络模型也有一些妥协。目前还不太清楚神经网络模型对结构有什么了解。此外，通常假设有大量带注释的训练数据随时可用，但对于许多语言和领域（医疗保健）来说，这种假设是不现实的。在这个项目中，我们的目标是重新审视早期结构化方法的好处，并将它们与神经网络模型的有益特征相结合。我们能否利用结构感知的神经模型，用更少的数据获得更有效的模型？为此，我们将努力更好地理解神经网络模型如何有效地捕获结构，尽管大多数神经模型并没有明确地对其建模。此外，我们将比较在神经模型中嵌入结构和约束的方法，并对比这些模型对可用训练数据量的依赖性。

Building effective and efficient models from data that map input structures to output structures is the lifeblood of Statistical Natural Language Processing (NLP). As an example, machine translation maps an input sequence in one language to an output sequence in another language, while syntactic and semantic analysers map input sequences to graphs, which express the structure of the sentence. In all of these cases, outputs that are predicted for a subset of an input sequence severely affect which remaining predictions are plausible. Namely, the outputs are structured.Neural network models have brought great advances to NLP modelling, since they boast of multiple alluring properties, such as scalable learning on web-scale datasets, learning reusable linguistic representations from raw text and having the ability to combine submodules from multiple modalities (text, image, speech) to learn complex mappings from inputs to outputs. However, the adoption of neural network models for NLP has come with some compromises. It is less clear what neural network models learn about structure. Furthermore, it is generally assumed that a large amount of annotated training data is readily available, an assumption that for many languages and domains (healthcare) is not realistic.In this project we aim to revisit the benefits of earlier structured approaches and intersect them with the beneficial traits of neural network models. Can we leverage neural models that are structure-aware to obtain effective and efficient models using less data? To this end, we will strive to better understand how neural network models effectively capture structure, despite the fact that most neural models don't explicitly model it. In addition, we will compare approaches that embed structure and constraints in neural models and contrast the dependence of these models on the amount of available training data.