Exploring Better Distributed Representation and Composition Models for Semantics

探索更好的分布式语义表示和组合模型

• 批准号: RGPIN-2018-06415
• 负责人: Zhu, Xiaodan
• 金额: $ 4.37万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=669897
• 关键词: Exploring; Better; Distributed; Representation; Composition

Computers understand very little of the meaning of human language. However, language data permeates almost all aspects of our daily life, e.g., news, emails, social media, telephone conversations, medical records, books, just to name a few. Search engines like Google are only scratching the surface of human language, and yet the impact on society and the economy is already immense. Developing better semantic modelling technologies will have extensive impact on real-life applications. In general, machine understanding of natural language with human-level performance remains one of the grand challenges of artificial intelligence (AI), where modelling meaning is a major obstacle to achieving the goal. ****** The long-term objective of this program is to devise state-of-the-art computational models that learn better representations for text. Specifically, we focus on distributed representations, including those learned with neural network models, in which text is represented as real-valued vectors. If properly learned, such representations have been shown to be very effective, resulting in cutting-edge performance on a wide range of natural language processing (NLP) problems.****** Building on our past contributions, this research program comprises three specific sets of short-term objectives for advancing distributed representations and neural network models for semantic representation. The first set aims to contribute novel algorithms for semantic composition. The principle of compositionality, positing that the meaning of a whole is a (complicated) function of the meaning of the parts, is a fundamental approach to learning representation for natural language. Compositionality is in general regarded by many as a basic ingredient of human intelligence. Based on our recent progress, this proposal aims to explore better neural-network-based semantic composition by further investigating external knowledge. In this set of objectives, we will also further explore non-compositionality, a basic phenomenon in language, in neural composition models. In the second set of objectives, we will delve into several specific aspects of semantics to deepen our understanding of distributed representation and neural-network-based modelling. While distributed representation has been shown to be very effective for modelling similarity and relatedness, its effectiveness for representing other key aspects of semantics, such as contrasting meaning and entailment, requires more investigation. The third set of objectives attempt to learn better distributed representation by further exploring syntactic structures. Language is rich in structures. It still remains a widely open question as to how to effectively consider such structures with the demonstrated modelling ability of distributed representation and neural networks.

计算机对人类语言的意义理解甚少。然而，语言数据几乎渗透到我们日常生活的方方面面，例如新闻、电子邮件、社交媒体、电话交谈、医疗记录、书籍等等。像谷歌这样的搜索引擎只是触及了人类语言的皮毛，但它对社会和经济的影响已经是巨大的。开发更好的语义建模技术将对现实生活中的应用产生广泛影响。总的来说，对自然语言的机器理解和人类水平的表现仍然是人工智能(AI)的重大挑战之一，建模意义是实现这一目标的主要障碍。*该计划的长期目标是设计最先进的计算模型，以学习更好的文本表示法。具体地说，我们关注分布式表示法，包括那些通过神经网络模型学习的表示法，其中文本表示为实值向量。如果学习得当，这种表示法已被证明是非常有效的，导致了在广泛的自然语言处理(NLP)问题上的前沿表现。*基于我们过去的贡献，这项研究计划包括三组特定的短期目标，用于推进分布式表示法和用于语义表示的神经网络模型。第一组旨在为语义合成贡献新的算法。构成性原则假定整体的意义是部分意义的(复杂)函数，是学习自然语言表征的基本途径。合成能力通常被许多人视为人类智力的一个基本要素。基于我们的最新进展，该建议旨在通过进一步研究外部知识来探索更好的基于神经网络的语义组合。在这组目标中，我们还将进一步探讨神经合成模型中的非合成性，这是语言中的一种基本现象。在第二组目标中，我们将深入研究语义的几个具体方面，以加深我们对分布式表示和基于神经网络的建模的理解。虽然分布式表示被证明在建模相似性和关联性方面非常有效，但它在表示语义的其他关键方面的有效性，如对比意义和蕴涵，还需要更多的研究。第三组目标试图通过进一步探索句法结构来学习更好的分布式表示法。语言具有丰富的结构。如何有效地考虑这种结构，并展示分布式表示和神经网络的建模能力，仍然是一个广泛悬而未决的问题。