SHF: Small: Addressing Challenges for the Next Decade of Massively Parallel NUMA Accelerators

在半导体行业中，使Dennard比例和摩尔定律得以实现的物理和经济原理已经达到了它们的临界点。然而，随着在单个芯片平台上经济地制造出越来越多的晶体管，处理器行业已经转向创建由多个被称为芯片的子组件组成的单封装计算系统。Chiplet通过高带宽封装网络进行通信，为下一个十年的透明性能扩展提供了潜力。然而，小芯片将具有挑战性的非均匀存储器访问特性引入了传统上不受这些影响的单封装系统。该项目开发了一些技术，以克服高性能单封装和多封装系统上非统一内存访问的挑战，而无需程序员干预。探索程序员透明的伸缩机制提高了程序的可移植性和生命周期，降低了软件的成本和复杂性。通过创建课程内容和本科生暑期实习，该项目促进了对后摩尔时代机器编程的理解，以及随着系统复杂性的增加，如何设计计算加速器以将对最终程序员的影响降至最低。该项目开发了协调的数据放置和线程调度算法，利用来自编译器的静态信息和来自运行时系统的动态信息来通知数据放置和基于硬件的线程调度。它通过开发具有分层互连的开源图形处理单元(GPU)仿真器来推进最先进的技术，该仿真器可用于对基于芯片的GPU和多GPU系统进行建模。研究人员正在探索在GPU中进行编译器知情的数据放置和线程调度。初步结果表明，对代码的静态分析可以预测GPU线程块访问的数据。分析表明，通过构建新的静态技术来确定网格中的哪些线程共享内存页面以及共享的方式是可能的，这些技术为数十年来针对顺序代码的编译器工作增加了一个额外的维度。利用静态信息，结合GPU驱动程序提供的运行时信息，研究人员正在开发高级数据放置、预取和线程调度算法。未来基于芯片的设计和现有的多GPU系统都受益于这些算法的发展。展望当今GPU中使用的高带宽内存，该项目探索了基于芯片的系统中异类内存的系统级含义。在使用高带宽存储器、传统的动态随机存取存储器和非易失性存储器的未来GPU系统中，数据放置和线程调度更加重要。这种系统中的问题大小预计会如此之大，以至于机会性数据放置和线程调度甚至比传统系统中更关键。该项目使用基于机器学习工作负载的内核内生产者-消费者本质的共享模式来改变程序的代码布局、运行时数据放置和线程块调度算法，以最大化多节点系统中的局部性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The physical and economic principles that enabled Dennard scaling and Moore's law in the semiconductor industry have reached their breaking point. However, as the number of transistors economically fabricated on a single chip plateaus, the processor industry has pivoted to create single-package computing systems, composed of multiple sub-components known as chiplets. Chiplets, which communicate via high-bandwidth on-package networks, offer the potential for transparent performance scaling into the next decade. However, chiplets introduce challenging non-uniform memory access characteristics into single-package systems that have traditionally not been subject to these effects. This project develops techniques to overcome the challenges of non-uniform memory accesses on high-performance single- and multi-package systems without programmer intervention. Exploring programmer-transparent scaling mechanisms improves the portability and lifetime of programs, decreasing the cost and complexity of software. Through the creation of course content and undergraduate summer internships, the project fosters an understanding of how to program machines in a post-Moore world and how compute accelerators should be designed to minimize the impact on the end-programmer as system complexity increases.This project develops coordinated data placement and thread scheduling algorithms that leverage static information from the compiler and dynamic information from the runtime system to inform data placement and hardware-based thread scheduling. It advances the state-of-the-art by developing an open-source Graphic Processing Unit (GPU) simulator with a hierarchical interconnect that can be used to model both chiplet-based GPUs and multi-GPU systems. The researchers are exploring compiler informed data placement and thread scheduling in GPUs. Initial results demonstrate that a static analysis of the code can predict the data accessed by GPU threadblocks. Analysis shows that it is possible to determine which threads in a grid share memory pages, and the manner of that sharing, by building new static techniques that add an additional dimension to decades of work on compilers for sequential code. Using static information, in combination with runtime information provided by GPU drivers, the researchers are developing advanced data placement, prefetching, and thread scheduling algorithms. Both future chiplet-based designs and existing multi-GPU systems benefit from the development of these algorithms. Looking beyond the high-bandwidth memory used in GPUs today the project explores the system-level implications of heterogeneous memory in a chiplet-based system. Data placement and thread scheduling have even more importance in GPU systems of the future that make use of high bandwidth memory, traditional dynamic random-access memory, and non-volatile memory. The problem sizes in such systems are anticipated to be so large that opportunistic data placement and thread scheduling are even more critical than in conventional systems. The project uses sharing patterns based on the inter-kernel producer-consumer nature of machine learning workloads to change the program's code layout, runtime data placement, and threadblock scheduling algorithm to maximize locality in multi-node systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.