Exploring Better Distributed Representation and Composition Models for Semantics

探索更好的分布式语义表示和组合模型

• 批准号: RGPIN-2018-06415
• 负责人: Zhu, Xiaodan
• 金额: $ 4.37万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=669897
• 关键词: Exploring; Better; Distributed; Representation; Composition

Computers understand very little of the meaning of human language. However, language data permeates almost all aspects of our daily life, e.g., news, emails, social media, telephone conversations, medical records, books, just to name a few. Search engines like Google are only scratching the surface of human language, and yet the impact on society and the economy is already immense. Developing better semantic modelling technologies will have extensive impact on real-life applications. In general, machine understanding of natural language with human-level performance remains one of the grand challenges of artificial intelligence (AI), where modelling meaning is a major obstacle to achieving the goal. ****** The long-term objective of this program is to devise state-of-the-art computational models that learn better representations for text. Specifically, we focus on distributed representations, including those learned with neural network models, in which text is represented as real-valued vectors. If properly learned, such representations have been shown to be very effective, resulting in cutting-edge performance on a wide range of natural language processing (NLP) problems.****** Building on our past contributions, this research program comprises three specific sets of short-term objectives for advancing distributed representations and neural network models for semantic representation. The first set aims to contribute novel algorithms for semantic composition. The principle of compositionality, positing that the meaning of a whole is a (complicated) function of the meaning of the parts, is a fundamental approach to learning representation for natural language. Compositionality is in general regarded by many as a basic ingredient of human intelligence. Based on our recent progress, this proposal aims to explore better neural-network-based semantic composition by further investigating external knowledge. In this set of objectives, we will also further explore non-compositionality, a basic phenomenon in language, in neural composition models. In the second set of objectives, we will delve into several specific aspects of semantics to deepen our understanding of distributed representation and neural-network-based modelling. While distributed representation has been shown to be very effective for modelling similarity and relatedness, its effectiveness for representing other key aspects of semantics, such as contrasting meaning and entailment, requires more investigation. The third set of objectives attempt to learn better distributed representation by further exploring syntactic structures. Language is rich in structures. It still remains a widely open question as to how to effectively consider such structures with the demonstrated modelling ability of distributed representation and neural networks.

计算机对人类语言的理解非常有限。然而，语言数据几乎渗透到我们日常生活的各个方面，例如，新闻，电子邮件，社交媒体，电话交谈，医疗记录，书籍，仅举几例。像谷歌这样的搜索引擎只触及人类语言的表面，但对社会和经济的影响已经是巨大的。开发更好的语义建模技术将对实际应用产生广泛的影响。总的来说，机器理解自然语言与人类水平的性能仍然是人工智能（AI）的重大挑战之一，其中建模意义是实现目标的主要障碍。****** 该计划的长期目标是设计最先进的计算模型，以学习更好的文本表示。具体来说，我们专注于分布式表示，包括那些用神经网络模型学习的表示，其中文本表示为实值向量。如果学习得当，这种表示已被证明是非常有效的，从而在广泛的自然语言处理（NLP）问题上产生尖端性能。 基于我们过去的贡献，该研究计划包括三组特定的短期目标，用于推进分布式表示和语义表示的神经网络模型。第一组的目的是贡献新的算法的语义组成。组合性原则，假定整体的意义是部分意义的（复杂）函数，是学习自然语言表征的基本方法。组合性通常被许多人认为是人类智力的基本组成部分。基于我们最近的进展，该建议旨在通过进一步研究外部知识来探索更好的基于神经网络的语义组合。在这组目标中，我们还将进一步探索非组合性，这是语言中的一种基本现象，在神经组合模型中。在第二组目标中，我们将深入研究语义的几个具体方面，以加深我们对分布式表示和基于神经网络的建模的理解。虽然分布式表示已被证明是非常有效的建模相似性和相关性，其有效性表示语义的其他关键方面，如对比的意义和蕴涵，需要更多的调查。第三组目标试图通过进一步探索句法结构来学习更好的分布式表示。语言具有丰富的结构。它仍然是一个广泛开放的问题，如何有效地考虑这种结构与分布式表示和神经网络的建模能力。