SHF:Small:Software and Hardware Optimizations for Learning over Graphs

许多来自国家重要领域的新兴高性能应用程序（例如，天体物理学，计算化学，药物发现和核物理学）使用大型图结构（数百万个节点和边连接它们）来表示它们的数据，并对它们执行不同类型的分析，目的是从海量数据中理解和提取有用的信息。虽然这种基于图形的分析正在迅速成为高性能计算的基本组成部分，但现有的图形分析方法不能随着硬件资源的增加而扩展，主要采用传统的机器学习/图形分析策略，并且没有充分利用新兴的异构计算和存储元素。因此，从大规模基于图形的数据中获得洞察力的时间大大增加，从而减慢了科学发现的速度。受这一观察结果的启发，这个由nsf资助的项目从整体的角度探索了图神经网络（gnn），这是一种直接在图结构上操作的神经网络，作为优化各种高性能应用程序的主要工具，这些应用程序受益于机器学习和数据分析。这个项目的最终目标是从现有的机器学习机制到gnn在各种应用领域的顺利和有效的过渡。通过促进更高效、更经济地使用定制集群、超级计算机和云系统提供的硬件资源，该项目还有望为广大研究人员、从业者和机器学习公司降低进入GNN世界的门槛。本研究的教育和推广部分包括：1)通过垂直整合的研究项目，本科生参与；2)开设GNNs研究生新课程；3)参加宾夕法尼亚州立大学的science - u项目（面向K-12年级学生的暑期科学营）；4)为高中女生和高中教师举办的暑期讲习班。更具体地说，该项目：1)探索gnn的理论基础，目标是确定收敛和可扩展性问题的根源，这些问题对于在高性能应用中使用gnn至关重要；2)研究一种支持GNN计算的与体系结构无关的编程语言，特别关注开发人员的生产力、语言表现力和可扩展性的便利性（以确保它与现有的编程范例、模型和工具互操作）；3)探索编译器对自动优化和将GNN应用映射到新兴硬件平台（包括多核cpu、gpu和fpga及其集成）的支持；4)为GNN计算开发定制架构支持，目标是超越当前可编程硬件选项的性能；5)对基于gnn的应用进行端到端实验评估，以确定需要进一步关注的方面；最后6)开发了一个基于gnn的基准测试套件，可以在各种硬件平台上使用。这六个组件共同形成了一个多层生态系统，用于理解和优化可以从图驱动学习中受益的高性能、大规模应用程序。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many emerging high-performance applications from various domains of national importance (e.g., astrophysics, computational chemistry, drug discovery, and nuclear physics) employ large graph structures (with millions of nodes and edges connecting them) to represent their data, and perform different types of analytics on them, with the goal of understanding and extracting useful information from this massive data. While this graph-based analytics is fast becoming a fundamental piece of high-performance computing, existing analytics methods on graphs do not scale with increased hardware resources, mostly employ conventional machine learning/graph analysis strategies, and do not take full advantage of emerging heterogeneous compute and storage elements. As a result, the time-to-insight from large-scale graph-based data increases significantly, thereby slowing down scientific discoveries. Motivated by this observation, this NSF-funded project explores, from a holistic viewpoint, Graph Neural Networks (GNNs), a type of Neural Network which directly operates on graph structures, as a main tool to optimize various high-performance applications that benefit from machine learning and data analytics. This project has the ultimate goal of making the transitioning from existing machine learning mechanisms to GNNs smooth and effective in a variety of application domains. By facilitating more efficient and cost-effective use of hardware resources that are provided by custom clusters, supercomputers and cloud systems, this project is also expected to reduce the barrier to entry to the GNN world for a broad population of researchers, practitioners, and machine learning companies. The educational and outreach components of this research include 1) undergraduate student involvement via vertically integrated research projects; 2) a new graduate course on GNNs; 3) participation of Science-U program (a summer science camp for K-12) at Penn State, and 4) summer workshops for high school girls and high school teachers.More specifically, this project: 1) explores the theoretical foundations of GNNs with the goal of identifying the roots of convergence and scalability problems, which are critical to address when employing them in high-performance applications; 2) investigates an architecture-agnostic programming language support for GNN computations, focusing in particular on developer productivity, language expressiveness, and ease of extensibility (to ensure that it inter-operates with existing programming paradigms, models ,and tools); 3) explores compiler support for automatically optimizing and mapping GNN applications onto emerging hardware platforms (including multicore CPUs, GPUs, and FPGAs as well as their ensembles); 4) develops custom architecture support for GNN computations, with the goal of exceeding the performance of current programmable hardware options; 5) carries out an end-to-end experimental evaluation of GNN-based applications to identify the aspects that require further attention; and finally 6) develops a GNN-based benchmark suite that can be used on a wide variety of hardware platforms. These six components collectively form a multi-layer ecosystem tuned to understand and optimize high-performance, large-scale applications that can benefit from graph-driven learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.