III: Small: A New Machine Learning Paradigm Towards Effective yet Efficient Foundation Graph Learning Models

III：小型：一种新的机器学习范式，实现有效且高效的基础图学习模型

• 批准号: 2321504
• 负责人: Yanfang Ye
• 金额: $ 59.96万
• 依托单位: University of Notre Dame
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2321504&HistoricalAwards=false
• 关键词: III; Small; New; Machine; Learning

Inspired by the success of foundation language models in applications such as ChatGPT, one can envision the far-reaching impacts that can be brought by a pre-trained Foundation Graph Learning Model (FGLM) with broader applications in the areas such as scientific research, social network analysis, anomaly detection, drug discovery, and e-commerce. Despite the significant progress of pre-trained graph neural networks, there has not yet a FGLM that can achieve desired performance on various graph-learning-related tasks. To bridge this gap, the goal of this project is to design and develop a new machine-learning paradigm (techniques, methods, and models) towards effective yet efficient FGLMs, which will help researchers and practitioners advance their work in a wide range of real-world applications driven by the prevalent graph-structured data, thus helping to enhance national safety, public health, and welfare. The project outcomes (such as open-source code, benchmark data, and models) will be made publicly accessible and be broadly distributed through demos, publications, media presentations, and the like. This project will integrate research with education, including novel curriculum development, student mentoring, professional training and workforce development, and K-12 outreach activities aimed at women and underrepresented groups.By developing a new machine-learning paradigm to jointly solve the multi-task, cross-graph, and cross-domain challenges in graph learning at the first attempt, this project includes three interconnected research components towards effective yet efficient FGLMs. First, to realize the strong and consistent task-generalization ability for FGLMs in an effective yet affordable way, given a graph, the team will design and develop a new multi-task self-supervised graph learning framework with a novel multi-gradient descent optimization algorithm coupled with adaptive data augmentation to learn each task equitably well. Second, as real-world graphs are always incomplete, to learn comprehensive knowledge for a specific domain, the team will develop a new multi-graph co-training framework, specifically a variational expectation-maximization framework with relational knowledge distillation, to jointly train the generated graphs in an effective yet efficient manner while tackling the challenge of diversified node attributes of different graphs. Third, to further enable cross-domain knowledge transfer for pre-trained FGLMs, the team will develop mixture-of-expert based meta-learning techniques to characterize the latent properties of graphs from different domains and adaptively utilize the knowledge learned from existing domains to infer on a rarely seen or unseen domain. The new machine-learning paradigm developed in this project will accelerate the development in the rapidly evolving area of pre-trained graph neural networks, advance the field of information integration and informatics, and help researchers and practitioners in different domains to advance their work in a variety of real-world applications driven by the ubiquitous graph data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

受基础语言模型在诸如CHATGPT等应用程序中的成功的启发，人们可以设想预先训练的基础图形学习模型（FGLM）可以带来的深远影响，并在科学研究，社交网络分析，异常检测，药物发现和电子商务等领域具有更广泛的应用。尽管预先训练的图神经网络取得了重大进展，但尚未在各种与图形学习相关的任务上实现所需性能的FGLM。为了弥合这一差距，该项目的目标是设计和开发新的机器学习范式（技术，方法和模型）朝着有效但有效的FGLMS朝着高效而有效的FGLM，这将有助于研究人员和从业人员在广泛的图形结构数据中驱动的广泛的现实应用程序中推进他们的工作，从而有助于增强国家安全，公共卫生，公共卫生，保健，并提供公共卫生，努力，并提供全国性的保健，并提供全国安全。项目结果（例如开源代码，基准数据和模型）将被公开访问，并通过演示，出版物，媒体演示等广泛分发。该项目将与教育相结合，包括新的课程发展，学生指导，专业培训和劳动力发展以及针对女性和代表性不足的小组的K-12外展活动。开发一个新的机器学习范式以共同解决了第一次尝试的互联网研究，在第一次尝试中，在第一次尝试中，将多任务，交叉杂志和跨案例挑战挑战，包括三个Intersconnement componnect for for componnement compons componnect componnect componnef compyn for for componnectn comment for componnef for componnef for componnef。首先，为了以有效但负担得起的方式实现FGLM的强大而一致的任务将来，鉴于图，团队将使用新型的多任务自我监督的图形学习框架来设计和开发新型的多层次下降下降量优化算法，并与适应性数据增强相结合，以学习每个任务。其次，由于现实世界图总是不完整的，要了解特定领域的全面知识，团队将开发一个新的多画图共同训练框架，尤其是一个具有关系知识蒸馏的各种期望最大化的框架，以有效但有效的方式共同训练生成的图形，同时又可以解决不同图表的不同图表的挑战。第三，为了进一步为预训练的FGLM提供跨域知识传递，该团队将开发基于专家的元学习技术的混合物，以表征来自不同域的图形的潜在特性，并自适应地利用从现有域中学到的知识来推断出很少见到或看不见的域。该项目中开发的新机器学习范式将加速预先训练的图形神经网络快速发展的领域的发展，推进信息集成和信息学领域，并帮助不同领域的研究人员和实践者在各种现实的应用中推动其在各种真实的应用中推动其在ubiquitos dection dection dection dection dection dectif的现实应用中推动其工作的工作。基金会的智力优点和更广泛的影响评论标准。