Neural generative models for natural language

神经网络是强大的机器学习方法，它推动了自然语言处理（NLP）研究的边界，并导致了许多成功应用的开发。拟议的研究旨在开发用于受控自然语言生成的神经网络模型。在许多情况下，神经网络是黑箱，因为通常不可能解释模型学习的输入的内部（潜在）表示。控制生成的文本的特征的一种方法是学习人类可解释的表示。为此，我们需要解开输入文本的不同变化因素的学习潜在表征。在我们最近的研究中，我和我的学生开发了一个神经模型，成功地解开了文本的风格（情感）和内容的表征，并证明了它在文本风格迁移中的有效性。 可控文本生成是一个快速发展的研究领域，迫切需要开发神经模型以实现许多实际应用，包括对话系统，文本风格转换和摘要。我们的第一个目标是开发多类风格转换的模型。风格类别的例子是作者的风格或文本中表达的情感。风格转换的实际应用包括写作助手，它可以自动建议更好地遵循目标风格的用户编写文本的替代版本。这种模型的另一个应用是对话系统，它生成以特定风格、人物或情感为条件的响应。我们的第二个目标是开发神经模型，解开文本中句法和语义信息的潜在表征。前提是同一个概念在句法上可以用许多不同的方式来表达。这种分离将使我们能够通过控制生成句子的结构来执行更复杂的风格转换。这种模型的另一个重要含义是与当前模型相比，文本含义的潜在更好的表示学习，其在相同的潜在空间中编码句法和语义特征。这将对许多下游任务有实际好处，例如对自然语言理解至关重要的释义检测。第三个目标是学习较长文本序列（如段落和整个文档）的长期特征，并将其潜在表示与段落级表示分离。这项研究有几个潜在的影响，如保持整个文档中生成的句子的文体一致性，并生成连贯的多句子序列。拟议的研究将为六名研究生提供培训机会，他们将获得深度学习方面的最先进知识和实践技能，这是最热门的新兴就业领域之一。

Neural networks are powerful machine learning methods that have pushed the boundaries of research in Natural Language Processing (NLP) and led to the development of many successful applications. The proposed research is aimed at developing neural network models for controlled natural language generation. Neural networks are in many cases black boxes, as it is often impossible to interpret the internal (latent) representations of the input learned by the models. One way to control the characteristics of the generated text is to learn the representations that are interpretable by humans. For that we need to disentangle the learned latent representations of different factors of variations of the input text. In our recent research, my students and I developed a neural model that successfully disentangles the representation of style (sentiment) and content of text, and demonstrated its effectiveness in text style transfer. Controllable text generation is a fast developing research field, and there is an urgency to develop neural models to enable many practical applications, including dialogue systems, text style transfer and summarization. Our first objective is to develop models for multi-class style transfer. Examples of style categories are author's style or emotions expressed in text. Practical applications of style transfer include writing assistants that automatically suggest alternate versions of user-written text that adhere to the target style better. Another application of such models is dialogue systems, which generate responses conditioned on a specific style, persona or emotion. Our second objective is to develop neural models that disentangle the latent representations of syntactic and semantic information in text. The premise is that the same idea can be expressed in many different ways syntactically. This disentanglement will potentially allow us to perform more complex style transfer by controlling the structure of generated sentences. Another important implication of such models is a potentially better representation learning of text meaning compared to the current models, which encode both syntactic and semantic characteristics in the same latent space. This will have practical benefits for many downstream tasks, such as paraphrase detection, which is essential for natural language understanding. The third objective is to learn long-term features that are characteristic of longer text sequences, such as paragraphs and entire documents, and to disentangle their latent representation from the sentence-level representations. This research has several potential implications, such as maintaining stylistic consistency of generated sentences throughout the document, and generating coherent multi-sentence sequences. The proposed research will provide training opportunities for six graduate students, who will acquire state-of-the-art knowledge and practical skills in deep learning, which is one of the top emerging areas of employment.