III: Small: EAGER: Representation Learning of Connotation and Denotation Knowledge for Atomic Information Units

人类社会已经发展出许多复杂的大规模符号系统，如自然语言、逻辑、数学等，它们可以编码非常复杂的信息。虽然符号系统的主要应用和动机是在不同实体之间进行横向/空间（例如，演讲者在会议中进行演讲）和/或垂直/临时（例如，阅读历史书）的交流，但这些符号系统所代表的信息最终是由人类大脑创建，修改和处理/计算的，大量的神经网络在子符号层面处理信息。符号加工和亚符号加工之间的关系和联系是什么？支持符号级处理的子符号级内部结构和机制是什么？除了浅层技术（例如，自然语言处理中的字符串匹配）之外，符号系统是否有任何深度计算机制？所有这些问题都是多个研究领域和科学学科的基础，并吸引了几代研究人员和科学家，从早期的哲学外延和内涵研究到最近的语义空间构建研究。该项目将重点研究自然语言中单词的表示信息的建模和表示。该项目主要关注子符号层面的语义理解，将为自然语言和人类智能提供有价值的见解，为建立计算语言学、心理学、语言习得等领域的大规模测试平台铺平道路，并为许多科学领域带来广泛的跨学科影响。该项目包括一个精心设计的教育部分，它直接促进本科生和研究生的研究和培训，鼓励少数民族和妇女参与，并对计算机科学课程和课件产生可持续的影响，超出了本项目的范围。这个项目的总体目标是研究如何在现有的向量空间方法之外，用一个内部结构（例如，一个神经网络）来表示一个词，以支持更复杂的符号处理技术，而不是肤浅的字符串匹配。具体来说，这个项目有三个研究目标。第一个目标是研究表示单词内部结构的各种选择，这与神经结构搜索的活跃研究领域密切相关。在第二个目标中，由于自然语言的词汇量大，建模的内涵信息复杂，这些神经体系结构中的大量参数需要学习和调整，将开发一种自举方法来克服挑战许多深度学习模型的数据稀疏性问题。采用无监督方法，该项目的第三个目标是研究大规模知识获取的可行方法，这通常被认为是构建现实世界人工智能系统的严重障碍。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many complex large-scale symbolic systems have been developed by human society, such as natural languages, logic, mathematics, which can encode very complicated information. While the major application and motivation for symbolic systems is communication among different entities either horizontally/spatially (e.g., a speaker gives a presentation in a meeting) and/or vertically/temporarily (e.g., reading a history book), information represented by these symbolic systems are ultimately created, revised, and processed/computed by human brain, a large volume of neural network processing information at the sub-symbolic level. What is the relationship and connection between symbolic processing and sub-symbolic processing? What is the internal structure and mechanism at the sub-symbolic level that supports symbol-level processing? Is there any deep computation mechanism for symbolic systems beyond shallow techniques (e.g., string match in Natural Language Processing)? All of these questions are fundamental to multiple research fields and scientific disciplines, and have attracted researchers and scientists of many generations ranging from the early study of denotation and connotation in philosophy to more recent investigation of semantic space construction. This project will focus on modeling and representation of denotation information for words in a natural language. With the fundamental focus on understanding of semantics at the sub-symbolic level, this project will provide valuable insight to natural languages and human intelligence in general, pave the way to build a large-scale testbed for fields such as computational linguistics, psychology, language acquisition, and bring broad interdisciplinary impact on many scientific fields. This project includes a carefully-crafted education component, which directly promotes undergraduate and graduate research and training, encourages minority and woman participation, and has a sustainable impact on Computer Science curricula and courseware beyond the scope of this project.The overall goal of this project is to investigate how to represent a word with an internal structure (e.g., a neural network) beyond the existing approach of vector space to support more sophisticated symbolic processing techniques beyond shallow string matching. Specifically, there are three research objectives in this project. The first objective is to study the various options for representing internal structures of a word, which is closely related to the active research field of Neural Architecture Search. In the second objective, due to the large vocabulary size in a natural language and complex connotation information for modeling, a huge number of parameters in these neural architectures need to be learned and tuned, a bootstrapping approach will be developed to overcome the problem of data sparsity that challenges many deep learning models. With an unsupervised approach, the third objective of this project is to investigate a viable way for large-scale knowledge acquisition, which is generally recognized as a serious barrier for building real-world Artificial Intelligence systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.