SHF:Small:Software and Hardware Optimizations for Learning over Graphs

许多新兴的高性能应用程序来自各个具有国家重要性的领域(例如，天体物理、计算化学、药物发现和核物理)，它们使用大型图形结构(具有数百万个连接它们的节点和边)来表示它们的数据，并对它们执行不同类型的分析，目的是从这些海量数据中理解和提取有用的信息。虽然这种基于图形的分析正在迅速成为高性能计算的基础，但现有的图形分析方法没有随着硬件资源的增加而扩展，主要使用传统的机器学习/图形分析策略，并且没有充分利用新兴的异类计算和存储元素。因此，从基于图表的大规模数据中获得洞察的时间显著增加，从而减缓了科学发现的速度。受这一观察结果的启发，NSF资助的这个项目从整体的角度探索了图形神经网络(GNN)，这是一种直接在图形结构上操作的神经网络，作为优化各种受益于机器学习和数据分析的高性能应用程序的主要工具。该项目的最终目标是使现有机器学习机制在各种应用领域顺利和有效地过渡到GNN。通过促进更高效、更具成本效益地使用定制集群、超级计算机和云系统提供的硬件资源，该项目还有望降低广大研究人员、从业者和机器学习公司进入GNN世界的门槛。这项研究的教育和推广部分包括1)本科生通过垂直整合的研究项目参与；2)关于GNN的新的研究生课程；3)在宾夕法尼亚州立大学参加Science-U计划(K-12的夏季科学夏令营)，以及4)为高中女孩和高中教师举办的暑期讲习班。更具体地说，本项目：1)探索GNN的理论基础，目的是找出在高性能应用程序中使用GNN时关键要解决的收敛和可扩展性问题的根源；2)研究与体系结构无关的编程语言对GNN计算的支持，特别关注开发人员的生产力、语言表达能力和可扩展性的易用性(以确保它与现有的编程范例、模型和工具互操作)；3)探索编译器对自动优化GNN应用并将其映射到新兴硬件平台(包括多核CPU、GPU和FPGA以及它们的集成)的支持；4)开发对GNN计算的定制体系结构支持，目标是超越当前可编程硬件选项的性能；5)对基于GNN的应用进行端到端的实验评估，以确定需要进一步关注的方面；最后6)开发了一个基于GNN的基准测试套件，该套件可以在各种硬件平台上使用。这六个组成部分共同构成了一个多层生态系统，旨在了解和优化可受益于图形驱动学习的高性能、大规模应用程序。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Many emerging high-performance applications from various domains of national importance (e.g., astrophysics, computational chemistry, drug discovery, and nuclear physics) employ large graph structures (with millions of nodes and edges connecting them) to represent their data, and perform different types of analytics on them, with the goal of understanding and extracting useful information from this massive data. While this graph-based analytics is fast becoming a fundamental piece of high-performance computing, existing analytics methods on graphs do not scale with increased hardware resources, mostly employ conventional machine learning/graph analysis strategies, and do not take full advantage of emerging heterogeneous compute and storage elements. As a result, the time-to-insight from large-scale graph-based data increases significantly, thereby slowing down scientific discoveries. Motivated by this observation, this NSF-funded project explores, from a holistic viewpoint, Graph Neural Networks (GNNs), a type of Neural Network which directly operates on graph structures, as a main tool to optimize various high-performance applications that benefit from machine learning and data analytics. This project has the ultimate goal of making the transitioning from existing machine learning mechanisms to GNNs smooth and effective in a variety of application domains. By facilitating more efficient and cost-effective use of hardware resources that are provided by custom clusters, supercomputers and cloud systems, this project is also expected to reduce the barrier to entry to the GNN world for a broad population of researchers, practitioners, and machine learning companies. The educational and outreach components of this research include 1) undergraduate student involvement via vertically integrated research projects; 2) a new graduate course on GNNs; 3) participation of Science-U program (a summer science camp for K-12) at Penn State, and 4) summer workshops for high school girls and high school teachers.More specifically, this project: 1) explores the theoretical foundations of GNNs with the goal of identifying the roots of convergence and scalability problems, which are critical to address when employing them in high-performance applications; 2) investigates an architecture-agnostic programming language support for GNN computations, focusing in particular on developer productivity, language expressiveness, and ease of extensibility (to ensure that it inter-operates with existing programming paradigms, models ,and tools); 3) explores compiler support for automatically optimizing and mapping GNN applications onto emerging hardware platforms (including multicore CPUs, GPUs, and FPGAs as well as their ensembles); 4) develops custom architecture support for GNN computations, with the goal of exceeding the performance of current programmable hardware options; 5) carries out an end-to-end experimental evaluation of GNN-based applications to identify the aspects that require further attention; and finally 6) develops a GNN-based benchmark suite that can be used on a wide variety of hardware platforms. These six components collectively form a multi-layer ecosystem tuned to understand and optimize high-performance, large-scale applications that can benefit from graph-driven learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.