RTML: Large: Acceleration to Graph-Based Machine Learning

RTML：大型：加速基于图的机器学习

• 批准号: 1937599
• 负责人: Jason Cong
• 金额: $ 150万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1937599&HistoricalAwards=false
• 关键词: RTML; Large; Acceleration; Graph; Based

Graphs are ubiquitous, and often the fundamental data structure in many applications including bioinformatics, chemistry, healthcare, social networks, recommender systems and systems analysis. Machine learning (ML) using graphs is receiving increasing attention, both where graphs are a representation of data, as in graph neural networks (GNN) algorithms, and where graphs are an efficient ML model representation, as in arithmetic circuits representation of probabilistic graphical models. While useful, graph-based ML poses unique challenges to existing computation hardware (Central Processing Units and Graphics Processing Units) due to the combination of irregular memory access and dynamic parallelism imposed by the graph structure and the dense computation required for relevant learning algorithms, though hardware-based implementations are highly desirable to enable real-time processing of streams of data generated by such applications. The project addresses these challenges with a novel accelerator architecture for graph-based ML, along with a supporting open source software stack, simulator, and field-programmable gate-array (FPGA) prototype. Beyond the technical contributions, the project will integrate the latest research into several graduate and upper-division undergraduate courses. The project will also work with the UCLA Center for Excellence in Engineering and Diversity (CEED) and Women in Engineering to recruit highly diversified undergraduate and graduate students to participate in the research. The project targets a programmable and heterogeneous multi-accelerator architecture, with software-controlled compute and memory resources. It is specialized in the following ways to meet the needs of graph-based machine learning. First, it supports composing accelerator engines for efficient pipelining of graph-based prefetching with dense computation units. Second, the prefetching hardware will be co-designed with GNN algorithms to support recent and upcoming advances in graph sampling and graph-coarsening algorithms. Third, it will include a high bandwidth scratchpad architecture optimized for indirect access, and spatial compute fabrics (e.g. systolic arrays) optimized for dense computation. Finally, the execution model will be based on an architecture-aware task-parallel model, which has rich-enough primitives to take advantage of heterogeneous hardware, while being flexible enough to load balance for dynamic parallelism. The key components of the proposed architecture will be prototyped on an FPGA. Overall, the goal of the work is to greatly advance the state-of-the-art of graph-based ML in terms of model accuracy, efficiency, and real-time inference and learning. The project will also collaborate with a synergistic DARPA program for related hardware development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图是无处不在的，通常是许多应用中的基本数据结构，包括生物信息学、化学、医疗保健、社交网络、推荐系统和系统分析。使用图的机器学习(ML)正受到越来越多的关注，其中图是数据的表示，如在图神经网络(GNN)算法中，并且图是有效的ML模型表示，如在概率图形模型的算术电路表示中。尽管基于图的ML很有用，但由于图结构施加的不规则存储器访问和动态并行性的组合以及相关学习算法所需的密集计算，基于图的ML对现有的计算硬件(中央处理单元和图形处理单元)构成了独特的挑战，尽管基于硬件的实现非常希望能够实时处理由这种应用产生的数据流。该项目通过针对基于图形的ML的新型加速器架构，以及支持开源软件堆栈、模拟器和现场可编程门阵列(FPGA)原型来应对这些挑战。除了技术上的贡献，该项目还将把最新的研究整合到几门研究生和高级本科课程中。该项目还将与加州大学洛杉矶分校工程与多样性卓越中心(CEED)和工程女性合作，招募高度多元化的本科生和研究生参与研究。该项目的目标是一个可编程和异质的多加速器架构，具有软件控制的计算和内存资源。为了满足基于图的机器学习的需要，它在以下方面做了专门的工作。首先，它支持组合加速器引擎，以实现基于图的预取与密集计算单元的高效流水线。其次，预取硬件将与GNN算法共同设计，以支持图形采样和图形粗化算法的最新和即将到来的进步。第三，它将包括针对间接访问进行优化的高带宽便签式架构，以及针对密集计算进行优化的空间计算结构(例如脉动阵列)。最后，执行模型将基于体系结构感知的任务并行模型，该模型具有足够丰富的原语来利用异类硬件，同时具有足够灵活的负载平衡以实现动态并行。提出的体系结构的关键组件将在FPGA上进行原型制作。总体而言，这项工作的目标是在模型精度、效率以及实时推理和学习方面极大地促进基于图的ML的最新发展。该项目还将与DARPA相关硬件开发的协同计划合作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

DSAGEN: Synthesizing Programmable Spatial Accelerators

• DOI: 10.1109/isca45697.2020.00032
• 发表时间: 2020-05
• 期刊: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)
• 作者: Jian Weng;Sihao Liu;Vidushi Dadu;Zhengrong Wang;Preyas Shah;Tony Nowatzki

Automated Accelerator Optimization Aided by Graph Neural Networks

图神经网络辅助的自动加速器优化

• DOI: 10.1145/3490422.3502330
• 发表时间: 2022
• 期刊: Proceedings of the 59th ACM/IEEE Design Automation Conference (DAC
• 作者: Sohrabizadeh, Atefeh;Bai, Yunsheng;Sun, Yizhou;and Cong, Jason
• 通讯作者: and Cong, Jason

Extending High-Level Synthesis for Task-Parallel Programs

• DOI: 10.1109/fccm51124.2021.00032
• 发表时间: 2020-09
• 期刊: 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)
• 作者: Yuze Chi;Licheng Guo;Young-kyu Choi;Jie Wang;J. Cong

GStarX: Explaining Graph Neural Networks with Structure-Aware Cooperative Games

GStarX：用结构感知合作游戏解释图神经网络

• 发表时间: 2022
• 期刊: 36th Conference on Neural Information Processing Systems (NeurIPS 2022
• 作者: Zhang, Shichang;Liu, Yozen;Shah, Neil;Sun, Yizhou
• 通讯作者: Sun, Yizhou

HOPE: High-order Graph ODE For Modeling Interacting Dynamics

• 发表时间: 2023
• 作者: Xiao Luo;Jingyang Yuan;Zijie Huang;Huiyu Jiang;Yifang Qin;Wei Ju;Ming Zhang;Yizhou Sun