RTML: Large: Acceleration to Graph-Based Machine Learning

RTML：大型：加速基于图的机器学习

• 批准号: 1937599
• 负责人: Jason Cong
• 金额: $ 150万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1937599&HistoricalAwards=false
• 关键词: RTML; Large; Acceleration; Graph; Based

Graphs are ubiquitous, and often the fundamental data structure in many applications including bioinformatics, chemistry, healthcare, social networks, recommender systems and systems analysis. Machine learning (ML) using graphs is receiving increasing attention, both where graphs are a representation of data, as in graph neural networks (GNN) algorithms, and where graphs are an efficient ML model representation, as in arithmetic circuits representation of probabilistic graphical models. While useful, graph-based ML poses unique challenges to existing computation hardware (Central Processing Units and Graphics Processing Units) due to the combination of irregular memory access and dynamic parallelism imposed by the graph structure and the dense computation required for relevant learning algorithms, though hardware-based implementations are highly desirable to enable real-time processing of streams of data generated by such applications. The project addresses these challenges with a novel accelerator architecture for graph-based ML, along with a supporting open source software stack, simulator, and field-programmable gate-array (FPGA) prototype. Beyond the technical contributions, the project will integrate the latest research into several graduate and upper-division undergraduate courses. The project will also work with the UCLA Center for Excellence in Engineering and Diversity (CEED) and Women in Engineering to recruit highly diversified undergraduate and graduate students to participate in the research. The project targets a programmable and heterogeneous multi-accelerator architecture, with software-controlled compute and memory resources. It is specialized in the following ways to meet the needs of graph-based machine learning. First, it supports composing accelerator engines for efficient pipelining of graph-based prefetching with dense computation units. Second, the prefetching hardware will be co-designed with GNN algorithms to support recent and upcoming advances in graph sampling and graph-coarsening algorithms. Third, it will include a high bandwidth scratchpad architecture optimized for indirect access, and spatial compute fabrics (e.g. systolic arrays) optimized for dense computation. Finally, the execution model will be based on an architecture-aware task-parallel model, which has rich-enough primitives to take advantage of heterogeneous hardware, while being flexible enough to load balance for dynamic parallelism. The key components of the proposed architecture will be prototyped on an FPGA. Overall, the goal of the work is to greatly advance the state-of-the-art of graph-based ML in terms of model accuracy, efficiency, and real-time inference and learning. The project will also collaborate with a synergistic DARPA program for related hardware development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图无处不在，并且通常是许多应用程序的基本数据结构，包括生物信息学，化学，医疗保健，社交网络，推荐系统和系统分析。使用图的机器学习（ML）正受到越来越多的关注，其中图是数据的表示，如在图神经网络（GNN）算法中，图是有效的ML模型表示，如在概率图模型的算术电路表示中。虽然有用，但基于图的机器学习对现有的计算硬件（中央处理单元和图形处理单元）提出了独特的挑战，这是由于图结构强加的不规则内存访问和动态并行性以及相关学习算法所需的密集计算的结合，尽管基于硬件的实现非常需要能够实时处理这些应用程序生成的数据流。该项目采用了一种新颖的基于图形的机器学习加速器架构，以及支持的开源软件堆栈、模拟器和现场可编程门阵列（FPGA）原型，解决了这些挑战。除了技术贡献之外，该项目还将把最新的研究成果整合到几个研究生和高年级本科课程中。该项目还将与加州大学洛杉矶分校工程与多样性卓越中心（CEED）和工程女性合作，招募高度多元化的本科生和研究生参与研究。该项目的目标是一个可编程的异构多加速器架构，具有软件控制的计算和内存资源。它专门以以下方式满足基于图的机器学习的需求。首先，它支持组合加速器引擎，实现基于图的密集计算单元预取的高效流水线。其次，预取硬件将与GNN算法共同设计，以支持最近和即将出现的图采样和图粗化算法的进展。第三，它将包括为间接访问优化的高带宽刮擦板架构，以及为密集计算优化的空间计算结构（例如收缩阵列）。最后，执行模型将基于体系结构感知的任务并行模型，该模型具有足够丰富的原语以利用异构硬件，同时具有足够的灵活性以实现动态并行的负载平衡。所提出架构的关键组件将在FPGA上进行原型设计。总的来说，这项工作的目标是在模型准确性、效率、实时推理和学习方面大大提高基于图的机器学习的水平。该项目还将与DARPA的一个协同项目合作进行相关硬件开发。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

DSAGEN: Synthesizing Programmable Spatial Accelerators

• DOI: 10.1109/isca45697.2020.00032
• 发表时间: 2020-05
• 期刊: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)
• 作者: Jian Weng;Sihao Liu;Vidushi Dadu;Zhengrong Wang;Preyas Shah;Tony Nowatzki

Automated Accelerator Optimization Aided by Graph Neural Networks

图神经网络辅助的自动加速器优化

• DOI: 10.1145/3490422.3502330
• 发表时间: 2022
• 期刊: Proceedings of the 59th ACM/IEEE Design Automation Conference (DAC
• 作者: Sohrabizadeh, Atefeh;Bai, Yunsheng;Sun, Yizhou;and Cong, Jason
• 通讯作者: and Cong, Jason

Extending High-Level Synthesis for Task-Parallel Programs

• DOI: 10.1109/fccm51124.2021.00032
• 发表时间: 2020-09
• 期刊: 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)
• 作者: Yuze Chi;Licheng Guo;Young-kyu Choi;Jie Wang;J. Cong

GStarX: Explaining Graph Neural Networks with Structure-Aware Cooperative Games

GStarX：用结构感知合作游戏解释图神经网络

• 发表时间: 2022
• 期刊: 36th Conference on Neural Information Processing Systems (NeurIPS 2022
• 作者: Zhang, Shichang;Liu, Yozen;Shah, Neil;Sun, Yizhou
• 通讯作者: Sun, Yizhou

HOPE: High-order Graph ODE For Modeling Interacting Dynamics

• 发表时间: 2023
• 作者: Xiao Luo;Jingyang Yuan;Zijie Huang;Huiyu Jiang;Yifang Qin;Wei Ju;Ming Zhang;Yizhou Sun