CAREER: Reinventing Network-on-Chips of GPU-Accelerated Systems

图形处理单元(GPU)已迅速发展成为数据并行计算的高性能加速器。为了充分利用GPU的计算能力，片上网络需要提供及时的数据移动，以满足处理核对数据的请求。目前，快速增长的GPU处理核的处理能力与缓慢增长的片上网络带宽之间存在着很大的差距。正因为如此，GPU加速的系统以互连为主，片上网络成为其性能瓶颈。人工智能、图形分析和大数据等数据密集型应用的出现，给互联互通结构带来了更大的压力。此外，超越摩尔定律和Dennard Scaling的计算的未来正在走向新兴硬件体系结构的高级集成，例如张量芯和通过2.5D或3D堆叠的高带宽存储器。这种新的硬件架构也需要来自片上网络的有效支持。本研究的教育贡献包括：(1)制定有效的计算机系统教学策略，为学生创造一个积极的学习环境，并将所提出的研究成果整合到课程开发中；(2)将模块和模拟器整合到课程项目中，并指导学生进行计算机系统相关研究。本研究旨在为GPU加速系统改造片上网络，以消除通信瓶颈。该项目的一个主要成果是一套能够开发有效和高效的芯片上网络体系结构的技术。研究活动结合了系统建模、最先进的设计方法和尖端的VLSI技术。具体的研究目标是：(1)为GPU片上网络开发计算模型、基准和模拟器，目的是了解GPU应用程序，特别是最近出现的应用程序的通信行为；(2)开发针对GPU加速系统的创新网络架构，以提高性能和能效；(3)探索新兴的网络计算研究范式，以提高计算/通信效率。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Graphics processing units (GPUs) have rapidly evolved to become high-performance accelerators for data-parallel computing. To fully take advantage of the computing power of GPUs, on-chip networks need to provide timely data movement to satisfy the requests of data by the processing cores. Currently, there exists a big gap between the fast-growing processing power of the GPU processing cores and the slow-increasing on-chip network bandwidth. Because of this, GPU-accelerated systems are interconnect-dominated and the on-chip network becomes their performance bottleneck. The emergence of data-intensive applications, such as artificial intelligence, graph analysis, and big data is putting more pressure on the interconnection fabrics. Furthermore, the future of computing beyond Moore’s law and Dennard scaling is moving toward advanced integration of emerging hardware architectures, such as Tensor cores and high bandwidth memories through 2.5D or 3D stacking. Such new hardware architectures also call for effective support from the on-chip networks. The educational contributions of this research include: (1) develop effective strategies for teaching computer systems, create an active learning environment for students and integrate the proposed research results into the curriculum development; (2) integrate the modules and simulators into course projects and mentor students to conduct computer system related research.This research seeks to reinvent on-chip networks for GPU-accelerated systems to remove the communication bottleneck. A major outcome of the project is a set of techniques that enable the development of effective and efficient network-on-chip architectures. The research activities leverage a combination of system modeling, state-of-the-art design methods, and cutting-edge VLSI technologies. The specific research objectives are: (1) develop computational models, benchmarks, and simulator for GPU on-chip networks, with the goal of understanding the communication behavior of GPU applications, especially the recently emerged ones; (2) develop innovative network architectures tailored for GPU-accelerated systems for improved performance and power efficiency; (3) explore the emerging research paradigm of in-network computing to improve the computation/communication efficiency.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.