Exploring New Neural Computing Models for Natural Language Understanding

探索自然语言理解的新神经计算模型

• 批准号: RGPIN-2018-05870
• 负责人: Jiang, Hui
• 金额: $ 4.66万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=654658
• 关键词: Exploring; New; Neural; Computing; Models

In this research program, we aim to explore new neural computing models to address some deficiencies of the current deep learning approaches for natural language understanding. First of all, we will study new methods to perform effective unsupervised or semi-supervised learning of neural networks for language understanding. The current successes of deep learning approaches largely rely on the popular error back-propagation based supervised learning algorithm, which requires a large amount of labeled data as prerequisites for any effective learning. Unlike speech and vision, it is much more challenging to collect human-labeled data for any language understanding tasks. On the other hand, we may easily have access to tons of unlabeled text from many sources on the Web. A successful neural computing model for natural language understanding needs to use a more effective unsupervised learning method to bootstrap itself from these unlabeled data, and then fine-tune towards a specific understanding task using only a small amount of labeled data. This sort of semi-supervised learning is a promising and viable approach to build natural language understanding systems in the near future. Secondly, the current neural networks are quite weak on long-term memory mechanisms, which are crucial in understating natural language. A same sentence may have very different meanings when appearing in various contexts, and even worse, the exact understanding of plain text normally stems from the background knowledge, which is not part of given text. In many cases, the given text simply serves as a trigger to retrieve some background knowledge or long-term memory to output as understanding. We will explore a novel approach to combine the neural networks based connectionist models with traditional symbolic approaches to address this issue since the symbolic approaches have shown tremendous advantages in reasoning over knowledge. The neural networks will be used as a flexible and powerful model to link surface text to background knowledge sources for effective reasoning. At last, we propose to investigate an open-domain question answer task as the main test bed for our research on both how to perform effective unsupervised learning and to combine connectionist models with traditional symbolic approaches.

在这个研究项目中，我们的目标是探索新的神经计算模型，以解决当前深度学习方法在自然语言理解方面的一些不足。首先，我们将研究新的方法，对神经网络进行有效的无监督或半监督学习，用于语言理解。目前深度学习方法的成功很大程度上依赖于流行的基于误差反向传播的监督学习算法，该算法需要大量标记数据作为任何有效学习的先决条件。与语音和视觉不同，为任何语言理解任务收集人类标记的数据更具挑战性。另一方面，我们可以很容易地从网络上的许多来源访问大量未标记的文本。一个成功的用于自然语言理解的神经计算模型需要使用一种更有效的无监督学习方法从这些未标记的数据中自我引导，然后仅使用少量标记数据微调到特定的理解任务。这种半监督学习是在不久的将来建立自然语言理解系统的一种有前途和可行的方法。其次，目前的神经网络在长期记忆机制方面相当薄弱，而长期记忆机制对于理解自然语言至关重要。同一个句子出现在不同的上下文中可能会有非常不同的含义，更糟糕的是，对纯文本的准确理解通常源于背景知识，而背景知识不是给定文本的一部分。在许多情况下，给定的文本只是作为一个触发器来检索一些背景知识或长期记忆，以输出为理解。我们将探索一种新的方法，将基于神经网络的连接主义模型与传统的符号方法相结合来解决这个问题，因为符号方法在推理方面比知识表现出巨大的优势。神经网络将作为一种灵活而强大的模型，将表面文本与背景知识源连接起来，进行有效的推理。最后，我们建议研究一个开放域问答任务，作为我们研究如何进行有效的无监督学习以及将连接主义模型与传统符号方法结合起来的主要测试平台。