Neuro-symbolic graph-to-text generation

人们对能够使用真正的人类语言与用户交流的计算机越来越感兴趣。例如，智能家居设备与用户交谈，计算机自动撰写摘要或回答问题。这种交互包括用计算机生成人类语言的具有挑战性的步骤。这个项目促成了这样一个阶段：计算机已经知道要说什么，但不知道如何说。也就是说，计算机已经建立了它想要说的话的抽象版本，格式可以在内部使用，但现在它必须用语言来表达--选择单词和句子结构，并正确地使用语法。对于复杂的文本，目前的方法依赖于神经网络，这是一种强大的机器学习设备，基于大量数据训练，生成与人类惊人地相似的文本。然而，这些神经网络可以有自己的想法，丢弃或发明内容，因此生成的文本并不能准确地表达计算机的意思。神经网络也往往是不透明的，因为它们的内部工作基本上由大量难以解释的数字组成。在这个项目中，我将在这个过程中加入明确的语言结构。这将使模型更加透明，因为将有可解释的结构可供查看，解释模型的决策。它还将提供一个脚手架，神经生成系统可以据此生成句子，使其能够创建更接近计算机想要表达的内容的原始抽象表示形式的文本。两项核心技术贡献中的第一项是弄清楚语言结构和神经网络到底应该如何相互作用，以及它们如何最好地协同工作，以创建流畅、自然、具有准确含义的文本。第二个贡献是从只包含抽象表示对和对应句子的数据中学习语言结构的方法。这种配对是现场使用的标准数据类型，一种可以学习“隐藏的”语言结构的方法在实践中将非常有用。我还将创建一个可视化界面，以便我们可以实际查看模型是如何工作的。最后，我将在人机对话等应用程序中测试开发的方法，并对其进行优化。

There is a growing interest in computers that can communicate with users using real human language. For example, smart home devices talk with their users, and computers write automatic summaries or answer questions. This kind of interaction includes the challenging step of generating human language with a computer. This project contributes to the stage where the computer already knows what to say, but not how to say it. That is, the computer has built an abstract version of what it wants to say, in a format it can work with internally, but now it has to express it in language - choose words and sentence structure, and get the grammar right.For complex text, current methods rely on neural networks, powerful machine learning devices trained on large amounts of data, that generate surprisingly human-like text. However, these neural networks can have a mind of their own, dropping or inventing content, so that the generated text does not exactly express what the computer meant to say. Neural networks also tend to be intransparent, since their inner workings essentially consist of large amounts of hard-to-interpret numbers. In this project, I will add explicit linguistic structures into this process. This will make the model more transparent, since there will be interpretable structures to look at, explaining the model's decisions. It will also provide a scaffolding along which the neural generation system can generate the sentence, enabling it to create text that stays closer to the original, abstract representation of what the computer wanted to express.The first of two central technical contributions is to figure out how exactly the linguistic structures and the neural networks should interact, how they can best work together to create fluent, natural sounding text that has exactly the right meaning. The second contribution is a method to learn the linguistic structures from data that only contains pairs of abstract representations and corresponding sentences. Such pairs are the standard type of data used in the field, and a method that can learn the "hidden" linguistic structures will be very useful in practice.I will also create a visualization interface, so that we can actually look at how the model works. Finally, I will test the developed method in, and optimize it for, applications such as human-robot dialog.