Neural generative models for natural language

点击翻译按钮获取中文摘要

Neural networks are powerful machine learning methods that have pushed the boundaries of research in Natural Language Processing (NLP) and led to the development of many successful applications. The proposed research is aimed at developing neural network models for controlled natural language generation. Neural networks are in many cases black boxes, as it is often impossible to interpret the internal (latent) representations of the input learned by the models. One way to control the characteristics of the generated text is to learn the representations that are interpretable by humans. For that we need to disentangle the learned latent representations of different factors of variations of the input text. In our recent research, my students and I developed a neural model that successfully disentangles the representation of style (sentiment) and content of text, and demonstrated its effectiveness in text style transfer. Controllable text generation is a fast developing research field, and there is an urgency to develop neural models to enable many practical applications, including dialogue systems, text style transfer and summarization. Our first objective is to develop models for multi-class style transfer. Examples of style categories are author's style or emotions expressed in text. Practical applications of style transfer include writing assistants that automatically suggest alternate versions of user-written text that adhere to the target style better. Another application of such models is dialogue systems, which generate responses conditioned on a specific style, persona or emotion. Our second objective is to develop neural models that disentangle the latent representations of syntactic and semantic information in text. The premise is that the same idea can be expressed in many different ways syntactically. This disentanglement will potentially allow us to perform more complex style transfer by controlling the structure of generated sentences. Another important implication of such models is a potentially better representation learning of text meaning compared to the current models, which encode both syntactic and semantic characteristics in the same latent space. This will have practical benefits for many downstream tasks, such as paraphrase detection, which is essential for natural language understanding. The third objective is to learn long-term features that are characteristic of longer text sequences, such as paragraphs and entire documents, and to disentangle their latent representation from the sentence-level representations. This research has several potential implications, such as maintaining stylistic consistency of generated sentences throughout the document, and generating coherent multi-sentence sequences. The proposed research will provide training opportunities for six graduate students, who will acquire state-of-the-art knowledge and practical skills in deep learning, which is one of the top emerging areas of employment.