Neural generative models for natural language

自然语言的神经生成模型

• 批准号: RGPIN-2019-04897
• 负责人: Vechtomova, Olga
• 金额: $ 2.99万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=715224
• 关键词: Neural; generative; models; natural; language

Neural networks are powerful machine learning methods that have pushed the boundaries of research in Natural Language Processing (NLP) and led to the development of many successful applications. The proposed research is aimed at developing neural network models for controlled natural language generation. Neural networks are in many cases black boxes, as it is often impossible to interpret the internal (latent) representations of the input learned by the models. One way to control the characteristics of the generated text is to learn the representations that are interpretable by humans. For that we need to disentangle the learned latent representations of different factors of variations of the input text. In our recent research, my students and I developed a neural model that successfully disentangles the representation of style (sentiment) and content of text, and demonstrated its effectiveness in text style transfer. Controllable text generation is a fast developing research field, and there is an urgency to develop neural models to enable many practical applications, including dialogue systems, text style transfer and summarization. Our first objective is to develop models for multi-class style transfer. Examples of style categories are author's style or emotions expressed in text. Practical applications of style transfer include writing assistants that automatically suggest alternate versions of user-written text that adhere to the target style better. Another application of such models is dialogue systems, which generate responses conditioned on a specific style, persona or emotion. Our second objective is to develop neural models that disentangle the latent representations of syntactic and semantic information in text. The premise is that the same idea can be expressed in many different ways syntactically. This disentanglement will potentially allow us to perform more complex style transfer by controlling the structure of generated sentences. Another important implication of such models is a potentially better representation learning of text meaning compared to the current models, which encode both syntactic and semantic characteristics in the same latent space. This will have practical benefits for many downstream tasks, such as paraphrase detection, which is essential for natural language understanding. The third objective is to learn long-term features that are characteristic of longer text sequences, such as paragraphs and entire documents, and to disentangle their latent representation from the sentence-level representations. This research has several potential implications, such as maintaining stylistic consistency of generated sentences throughout the document, and generating coherent multi-sentence sequences. The proposed research will provide training opportunities for six graduate students, who will acquire state-of-the-art knowledge and practical skills in deep learning, which is one of the top emerging areas of employment.

神经网络是一种强大的机器学习方法，它推动了自然语言处理(NLP)的研究，并导致了许多成功的应用程序的开发。拟议的研究旨在开发用于受控自然语言生成的神经网络模型。在许多情况下，神经网络是黑匣子，因为它通常不可能解释由模型学习的输入的内部(潜在)表示。控制生成的文本的特征的一种方法是学习人类可解释的表示法。为此，我们需要将学习到的输入文本变化的不同因素的潜在表征分开。在我们最近的研究中，我和我的学生开发了一个神经模型，成功地分离了文本风格(情感)和内容的表征，并证明了其在文本风格迁移中的有效性。 可控文本生成是一个快速发展的研究领域，迫切需要开发神经模型来实现许多实际应用，包括对话系统、文本风格转换和摘要。 我们的第一个目标是开发多类风格迁移的模型。风格类别的例子是作者的风格或在文本中表达的情感。风格转换的实际应用包括写作助手，它可以自动建议用户书写的文本的替代版本，从而更好地坚持目标风格。这类模型的另一个应用是对话系统，它根据特定的风格、角色或情绪产生反应。 我们的第二个目标是开发神经模型，以分离文本中潜在的句法和语义信息表示。前提是，相同的想法可以用许多不同的方式在句法上表达。这种解开将潜在地允许我们通过控制生成的句子的结构来执行更复杂的风格转移。这种模型的另一个重要含义是，与当前的模型相比，这种模型可能会更好地表示文本意义，后者在相同的潜在空间中同时编码句法和语义特征。这将对许多下游任务产生实际好处，例如对自然语言理解至关重要的释义检测。 第三个目标是学习较长文本序列的长期特征，如段落和整个文档，并将其潜在表示从句子级别的表示中分离出来。这项研究有几个潜在的意义，例如在整个文档中保持生成句子的文体一致性，以及生成连贯的多句序列。 拟议的研究将为六名研究生提供培训机会，他们将获得深度学习方面的最新知识和实践技能，深度学习是最新出现的就业领域之一。