Neuro-symbolic graph-to-text generation

人们对能够使用真实的人类语言与用户交流的计算机越来越感兴趣。例如，智能家居设备与用户交谈，计算机自动编写摘要或回答问题。这种交互包括用计算机生成人类语言的挑战性步骤。这个项目有助于计算机已经知道该说什么，但不知道如何说。也就是说，计算机已经建立了它想要说的内容的抽象版本，以它可以内部工作的格式，但现在它必须用语言表达-选择单词和句子结构，并获得语法正确。对于复杂的文本，目前的方法依赖于神经网络，强大的机器学习设备在大量数据上进行训练，生成令人惊讶的类人文本。然而，这些神经网络可以有自己的想法，丢弃或发明内容，因此生成的文本并不能准确表达计算机的意思。神经网络也往往是不透明的，因为它们的内部工作基本上由大量难以解释的数字组成。在这个项目中，我将在这个过程中加入明确的语言结构。这将使模型更加透明，因为将有可解释的结构可以查看，解释模型的决策。它还将为神经生成系统生成句子提供一个沿着的框架，使其能够创建更接近计算机想要表达的内容的原始抽象表示的文本。两个核心技术贡献中的第一个是弄清楚语言结构和神经网络应该如何相互作用，它们如何最好地协同工作以创建流畅，自然发音的文本，具有正确的含义。第二个贡献是一种方法来学习的语言结构的数据，只包含对抽象表示和相应的句子。这样的对是该领域使用的标准数据类型，一种可以学习“隐藏”语言结构的方法在实践中将非常有用。我还将创建一个可视化界面，以便我们可以实际查看模型是如何工作的。最后，我将在人机对话等应用程序中测试所开发的方法，并对其进行优化。

There is a growing interest in computers that can communicate with users using real human language. For example, smart home devices talk with their users, and computers write automatic summaries or answer questions. This kind of interaction includes the challenging step of generating human language with a computer. This project contributes to the stage where the computer already knows what to say, but not how to say it. That is, the computer has built an abstract version of what it wants to say, in a format it can work with internally, but now it has to express it in language - choose words and sentence structure, and get the grammar right.For complex text, current methods rely on neural networks, powerful machine learning devices trained on large amounts of data, that generate surprisingly human-like text. However, these neural networks can have a mind of their own, dropping or inventing content, so that the generated text does not exactly express what the computer meant to say. Neural networks also tend to be intransparent, since their inner workings essentially consist of large amounts of hard-to-interpret numbers. In this project, I will add explicit linguistic structures into this process. This will make the model more transparent, since there will be interpretable structures to look at, explaining the model's decisions. It will also provide a scaffolding along which the neural generation system can generate the sentence, enabling it to create text that stays closer to the original, abstract representation of what the computer wanted to express.The first of two central technical contributions is to figure out how exactly the linguistic structures and the neural networks should interact, how they can best work together to create fluent, natural sounding text that has exactly the right meaning. The second contribution is a method to learn the linguistic structures from data that only contains pairs of abstract representations and corresponding sentences. Such pairs are the standard type of data used in the field, and a method that can learn the "hidden" linguistic structures will be very useful in practice.I will also create a visualization interface, so that we can actually look at how the model works. Finally, I will test the developed method in, and optimize it for, applications such as human-robot dialog.