III: Small: EAGER: Representation Learning of Connotation and Denotation Knowledge for Atomic Information Units

人类社会发展出了许多复杂的大规模符号系统，如自然语言、逻辑、数学等，它们可以对非常复杂的信息进行编码。虽然符号系统的主要应用和动机是不同实体之间的水平/空间通信（例如，发言者在会议中给出演示）和/或垂直地/临时地（例如，阅读历史书），这些符号系统所代表的信息最终由人脑（大量神经网络在亚符号层面处理信息）创建、修改和处理/计算。符号加工和次符号加工之间的关系和联系是什么？支持符号级处理的子符号级的内部结构和机制是什么？符号系统是否有超越浅层技术的深层计算机制（例如，自然语言处理中的字符串匹配）？所有这些问题都是多个研究领域和科学学科的基础，并吸引了许多代的研究人员和科学家，从哲学中的外延和内涵的早期研究到最近的语义空间构建研究。这个项目将集中在自然语言中单词的指称信息的建模和表示。该项目的基本重点是在子符号层面理解语义，将为自然语言和人类智能提供有价值的见解，为计算语言学，心理学，语言习得等领域建立大规模测试平台铺平道路，并对许多科学领域产生广泛的跨学科影响。该项目包括一个精心设计的教育部分，直接促进本科生和研究生的研究和培训，鼓励少数民族和妇女的参与，并对计算机科学课程和课件产生可持续的影响，超出了该项目的范围。神经网络）以支持超越浅串匹配的更复杂的符号处理技术。具体而言，本项目有三个研究目标。第一个目标是研究用于表示词的内部结构的各种选项，这与神经结构搜索的活跃研究领域密切相关。在第二个目标中，由于自然语言的词汇量很大，建模的内涵信息很复杂，这些神经架构中的大量参数需要学习和调整，将开发一种自举方法来克服挑战许多深度学习模型的数据稀疏问题。通过无监督的方法，该项目的第三个目标是研究大规模知识获取的可行方法，这通常被认为是构建现实世界人工智能系统的严重障碍。该奖项反映了NSF的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many complex large-scale symbolic systems have been developed by human society, such as natural languages, logic, mathematics, which can encode very complicated information. While the major application and motivation for symbolic systems is communication among different entities either horizontally/spatially (e.g., a speaker gives a presentation in a meeting) and/or vertically/temporarily (e.g., reading a history book), information represented by these symbolic systems are ultimately created, revised, and processed/computed by human brain, a large volume of neural network processing information at the sub-symbolic level. What is the relationship and connection between symbolic processing and sub-symbolic processing? What is the internal structure and mechanism at the sub-symbolic level that supports symbol-level processing? Is there any deep computation mechanism for symbolic systems beyond shallow techniques (e.g., string match in Natural Language Processing)? All of these questions are fundamental to multiple research fields and scientific disciplines, and have attracted researchers and scientists of many generations ranging from the early study of denotation and connotation in philosophy to more recent investigation of semantic space construction. This project will focus on modeling and representation of denotation information for words in a natural language. With the fundamental focus on understanding of semantics at the sub-symbolic level, this project will provide valuable insight to natural languages and human intelligence in general, pave the way to build a large-scale testbed for fields such as computational linguistics, psychology, language acquisition, and bring broad interdisciplinary impact on many scientific fields. This project includes a carefully-crafted education component, which directly promotes undergraduate and graduate research and training, encourages minority and woman participation, and has a sustainable impact on Computer Science curricula and courseware beyond the scope of this project.The overall goal of this project is to investigate how to represent a word with an internal structure (e.g., a neural network) beyond the existing approach of vector space to support more sophisticated symbolic processing techniques beyond shallow string matching. Specifically, there are three research objectives in this project. The first objective is to study the various options for representing internal structures of a word, which is closely related to the active research field of Neural Architecture Search. In the second objective, due to the large vocabulary size in a natural language and complex connotation information for modeling, a huge number of parameters in these neural architectures need to be learned and tuned, a bootstrapping approach will be developed to overcome the problem of data sparsity that challenges many deep learning models. With an unsupervised approach, the third objective of this project is to investigate a viable way for large-scale knowledge acquisition, which is generally recognized as a serious barrier for building real-world Artificial Intelligence systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.