权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Representation Learning via Variational Mean Field Theory

通过变分平均场理论进行表示学习

基本信息

批准号：
2401297
负责人：
Rongjie Lai
金额：
$ 74.98万
依托单位：
Purdue University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-10-01 至 2025-03-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2401297&HistoricalAwards=false
关键词：
Representation Learning via Variational Mean

项目摘要

Deep learning has had tremendous success in many science and engineering applications. It revolutionizes classical model-based methods using data-driven approaches in a computationally tractable way. Representation learning, which extracts useful information from raw data needed for downstream tasks (for example, classification, regression, and reinforcement learning), is one of the most important directions of deep learning. Despite its success in areas such as image and signal processing, speech and object recognition, natural language processing, chemistry and drug discovery, theoretical understanding of representation learning is far from satisfactory. This project aims at new understanding of representation learning (in particular, deep generative models and graph representation learning) using mean-field game (MFG) theory. The research is intended not only to provide a theoretical understanding of generative models and graph representations but also to serve as a key step in transforming deep representation learning from a black-box approach to an explainable and trustworthy method. The project will benefit researchers in academia, government labs, and industry and will provide interdisciplinary training in applied mathematics, engineering, and data science to undergraduate and graduate students. Collaboration with the MIT-IBM Watson AI Lab, which offers complementary technical skills and industrial angles, will enhance career opportunities for undergraduate and graduate students. This project aims at i) bridging theoretical and analytical tools in MFG with deep generative models and graph representation learning, and ii) exploring new data-driven architecture designs in MFG-guided representation learning through the lens of bi-level optimization. This first objective is to understand the practical normalizing flows as the variational MFG, where reversible particle trajectories in MFG can be naturally viewed as the generative and normalizing directions in normalizing flows. The second objective is to propose a new framework for graph representation learning based on MFG. This new way of modeling with graph-structured data overcomes the limitation of the message passing framework studied from the graph isomorphism angle, leading to new network architectures that are faster and more scalable to train. To complement the expert-based choice of the dependencies and architectures in MFG, a new bi-level optimization approach will be investigated to jointly learn model dependences, architectures, and parameters in the first two objectives. The complementary expertise of the research team is being leveraged to enrich the theoretical foundations of deep learning through MFG-, graph-, and optimization-based approaches. This research agenda is expected to foster multidisciplinary efforts at the intersection of representation learning, graph learning, bi-level optimization, signal processing, and control theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习在许多科学和工程应用中取得了巨大的成功。它以一种计算上易于处理的方式使用数据驱动的方法彻底改变了经典的基于模型的方法。表示学习从下游任务（例如分类、回归和强化学习）所需的原始数据中提取有用信息，是深度学习最重要的方向之一。尽管表征学习在图像和信号处理、语音和物体识别、自然语言处理、化学和药物发现等领域取得了成功，但对表征学习的理论理解还远远不能令人满意。该项目旨在使用平均场博弈（MFG）理论对表示学习（特别是深度生成模型和图表示学习）进行新的理解。该研究不仅旨在提供对生成模型和图形表示的理论理解，而且还将作为将深度表示学习从黑盒方法转变为可解释和可信赖方法的关键一步。该项目将使学术界、政府实验室和工业界的研究人员受益，并将为本科生和研究生提供应用数学、工程和数据科学方面的跨学科培训。与麻省理工学院-IBM沃森人工智能实验室合作，提供互补的技术技能和工业角度，将增加本科生和研究生的就业机会。该项目旨在i）将MFG中的理论和分析工具与深度生成模型和图形表示学习相结合，ii）通过双层优化的透镜，在MFG引导的表示学习中探索新的数据驱动架构设计。这第一个目标是理解实际的归一化流作为变分MFG，其中MFG中的可逆粒子轨迹可以自然地被视为归一化流中的生成和归一化方向。第二个目标是提出一个新的基于MFG的图表示学习框架。这种用图结构数据建模的新方法克服了从图同构角度研究的消息传递框架的局限性，从而产生了更快、更可扩展的新网络架构。为了补充MFG中基于专家的依赖关系和架构选择，将研究一种新的双层优化方法，以联合学习前两个目标中的模型依赖关系、架构和参数。研究团队的互补专业知识正在通过基于MFG，图形和优化的方法来丰富深度学习的理论基础。该研究议程预计将促进多学科的努力，在交叉的表示学习，图形学习，双层优化，信号处理和控制理论。该奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。