CAREER: A Novel Blueprint for Representation Learning of Relational Invariances

图、张量和逻辑公式是用于对复杂数据依赖进行建模的三个最基本的数学抽象。这些结构对于许多现代人工智能创新的设计至关重要，例如社交网络中的推荐系统、对环境进行推理的机器人、使用生成性模型设计新药，或者从数据中提取商业规则。在过去的一个世纪里，我们对图形和张量的理论理解有了显著的进步，现在有机会为日常的机器学习任务开发实用工具。该研究项目旨在使神经网络体系结构能够获得对复杂任务具有足够的表现力并且对于更多任务在计算上易于处理的数据表示。该项目利用PI以前的工作形成一个新的蓝图，以便更好地对复杂的关系输入数据进行建模，从而将集合、图、张量和逻辑公式的现代理论解释转化为经证明更有能力的实用工具。关键的洞察是利用不变量理论来开发一个新的框架，用于通过近似不变量来学习复杂关系数据的表示。这一提议框架的充分发展有望利用不变量理论和计算方法之间的协同作用。具体地说，本研究考察了：(A)通过遍历理论和变分逼近的最具表现力的易处理模型；(B)通过不变松弛和准不变神经结构来适应模型的可处理性和可表现性；以及(C)新的模型评估度量和不变表示的表现性测试。这个项目还探索了这些方法在定义在时间图和张量、集合和多集、超图和简单、逻辑公式和分子上定义的任务上的应用。该项目还将支持大量的外展活动，包括研讨会组织、课程开发和招募未被充分代表的少数族裔进入STEM职业生涯。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Graphs, tensors, and logical formulae are three of the most fundamental mathematical abstractions used to model complex data dependencies. These constructs are essential for the design of many modern AI innovations such as recommender systems in social networks, robots that reason about their environment, using generative models to design new drugs, or extracting business rules from data. Our theoretical understanding of graphs and tensors has significantly advanced in the past century, and the opportunity now exists for the development of practical tools for day-to-day machine-learning tasks. This research project aims to make available to neural network architectures data representations that are sufficiently expressive for complex tasks and computationally tractable for more tasks. This project leverages the PI’s prior work into a novel blueprint for better modeling complex relational input data, thereby translating the modern theoretical interpretation of sets, graphs, tensors, and logical formulae into provably more capable practical tools. The key insight is to leverage invariant theory to develop a novel framework for representation learning of complex relational data via approximate invariances. The full development of the framework of this proposal holds promise to harness the synergy between invariant theory and computational methods. Specifically, this study investigates: (a) most-expressive tractable models through ergodic theory and variational approximations; (b) adaptive model tractability and expressibility through invariance relaxations and quasi-invariant neural architectures; and (c) novel model evaluation metrics and tests of expressiveness for invariant representations. This project also explores the application of these methods on tasks defined on temporal graphs and tensors, sets and multisets, hypergraphs and simplices, logical formulae, and molecules. This project will also support substantial outreach activities, including workshop organization, course development, and the recruitment of underrepresented minorities to STEM careers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.