Exploring New Neural Computing Models for Natural Language Understanding

探索自然语言理解的新神经计算模型

• 批准号: RGPIN-2018-05870
• 负责人: Jiang, Hui
• 金额: $ 4.66万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=691231
• 关键词: Exploring; New; Neural; Computing; Models

In this research program, we aim to explore new neural computing models to address some deficiencies of the current deep learning approaches for natural language understanding. First of all, we will study new methods to perform effective unsupervised or semi-supervised learning of neural networks for language understanding. The current successes of deep learning approaches largely rely on the popular error back-propagation based supervised learning algorithm, which requires a large amount of labeled data as prerequisites for any effective learning. Unlike speech and vision, it is much more challenging to collect human-labeled data for any language understanding tasks. On the other hand, we may easily have access to tons of unlabeled text from many sources on the Web. A successful neural computing model for natural language understanding needs to use a more effective unsupervised learning method to bootstrap itself from these unlabeled data, and then fine-tune towards a specific understanding task using only a small amount of labeled data. This sort of semi-supervised learning is a promising and viable approach to build natural language understanding systems in the near future. Secondly, the current neural networks are quite weak on long-term memory mechanisms, which are crucial in understating natural language. A same sentence may have very different meanings when appearing in various contexts, and even worse, the exact understanding of plain text normally stems from the background knowledge, which is not part of given text. In many cases, the given text simply serves as a trigger to retrieve some background knowledge or long-term memory to output as understanding. We will explore a novel approach to combine the neural networks based connectionist models with traditional symbolic approaches to address this issue since the symbolic approaches have shown tremendous advantages in reasoning over knowledge. The neural networks will be used as a flexible and powerful model to link surface text to background knowledge sources for effective reasoning. At last, we propose to investigate an open-domain question answer task as the main test bed for our research on both how to perform effective unsupervised learning and to combine connectionist models with traditional symbolic approaches.

在这个研究项目中，我们的目标是探索新的神经计算模型，以解决目前用于自然语言理解的深度学习方法的一些不足。首先，我们将研究用于语言理解的神经网络进行有效的非监督或半监督学习的新方法。目前深度学习方法的成功在很大程度上依赖于流行的基于误差反向传播的监督学习算法，该算法需要大量的标记数据作为有效学习的前提。与语音和视觉不同，为任何语言理解任务收集人类标记的数据要困难得多。另一方面，我们可以很容易地从Web上的许多来源获得大量未标记的文本。一个成功的自然语言理解神经计算模型需要使用一种更有效的无监督学习方法来从这些未标记的数据中引导自己，然后仅使用少量的已标记的数据来针对特定的理解任务进行微调。这种半监督学习是在不久的将来建立自然语言理解系统的一种有前途和可行的方法。其次，目前的神经网络在长期记忆机制方面相当薄弱，而长期记忆机制是低估自然语言的关键。同一个句子在不同的上下文中出现时可能具有非常不同的含义，更糟糕的是，对纯文本的准确理解通常源于背景知识，而背景知识不是给定文本的一部分。在许多情况下，给定的文本只是作为触发器来检索一些背景知识或长期记忆，以输出为理解。我们将探索一种新的方法，将基于神经网络的连接主义模型与传统的符号方法相结合来解决这个问题，因为符号方法在推理方面显示出了相对于知识的巨大优势。神经网络将作为一种灵活而强大的模型，将表层文本与背景知识源联系起来，以进行有效的推理。最后，我们建议调查一个开放领域的问答任务，作为我们研究如何进行有效的无监督学习以及如何将连接主义模型与传统的符号方法相结合的主要测试平台。