CAREER: Compiler and Runtime Support for Multi-Tasking on Commodity GPUs

职业：商用 GPU 上多任务的编译器和运行时支持

• 批准号: 1750760
• 负责人: Bo Wu
• 金额: $ 50.15万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-05-01 至 2024-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1750760&HistoricalAwards=false
• 关键词: CAREER; Compiler; Runtime; Support; Multi

General-purpose Graphics Processing Units (GPU) computing has become mainstream, as witnessed in various domains such as machine learning, graph analytics, and scientific simulation. One notable trend is employing GPUs in data centers and cloud computing infrastructures to satisfy users' increasing demand to accelerate their applications. In such multi-tasking environments, applications from different users contend to use the shared GPU, leading to unpredictable and unacceptable performance degradation. This CAREER project aims at developing a set of compiler and runtime techniques to support multi-tasking on commodity GPUs in a transparent and efficient manner. The compiler techniques circumvent the hardware limitations to enable a set of features, such as preemption, and the runtime system schedules applications to utilize the potential of the GPU and guarantees quality of service. In addition, the investigator advances GPU education in the University to target both Computer Science (CS) and non-CS students based on a GPU education center.Specifically, the project investigates how to integrate compiler and runtime techniques to support multi-tasking on GPUs by building a system that achieves three goals. First, the system addresses GPU core contention by enabling flexible GPU kernel preemption. The compiler transforms the GPU program to be a preemptable form by circumventing the limitation imposed by the hardware thread scheduler. The runtime intercepts all GPU kernel launch requests and makes global preemption and scheduling decisions to maximize performance. Second, the system supports fine-grained sharing for threads from different applications to fully utilize hardware resources within GPU streaming multi-processors. The runtime guarantees the QoS of user-facing applications while optimizing overall throughput aided by performance prediction. Third, the system addresses GPU memory contention by coordinating GPU memory transfers, which considers memory access patterns and array reuse patterns.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

通用图形处理单元（GPU）计算已经成为主流，在机器学习、图形分析和科学模拟等各个领域都是如此。一个值得注意的趋势是在数据中心和云计算基础设施中使用gpu，以满足用户不断增长的加速应用程序的需求。在这种多任务环境中，来自不同用户的应用程序争用共享GPU，导致不可预测和不可接受的性能下降。这个CAREER项目旨在开发一套编译器和运行时技术，以透明和高效的方式支持商用gpu上的多任务处理。编译器技术绕过硬件限制来启用一组特性，例如抢占，运行时系统调度应用程序来利用GPU的潜力并保证服务质量。此外，研究者还推动了该大学的GPU教育，以GPU教育中心为基础，针对计算机科学（CS）和非CS学生。具体来说，该项目研究了如何通过构建一个实现三个目标的系统来集成编译器和运行时技术来支持gpu上的多任务处理。首先，该系统通过启用灵活的GPU内核抢占来解决GPU核心争用问题。编译器通过规避硬件线程调度器施加的限制，将GPU程序转换为可抢占的形式。运行时拦截所有GPU内核启动请求，并做出全局抢占和调度决策，以最大化性能。其次，系统支持来自不同应用程序的线程的细粒度共享，以充分利用GPU流多处理器内的硬件资源。运行时保证面向用户的应用程序的QoS，同时借助性能预测优化总体吞吐量。第三，系统通过协调GPU内存传输来解决GPU内存争用问题，其中考虑了内存访问模式和数组重用模式。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

DGSM: A GPU-Based Subgraph Isomorphism framework with DFS exploration

• DOI: 10.1109/rsdha56811.2022.00006
• 发表时间: 2022-11
• 期刊: 2022 IEEE/ACM Redefining Scalability for Diversely Heterogeneous Architectures Workshop (RSDHA)
• 作者: W. Han;Connor Holmes;Bo Wu

ELIχR: Eliminating Computation Redundancy in CNN-Based Video Processing

ELIÏR：消除基于 CNN 的视频处理中的计算冗余

• DOI: 10.1109/rsdha54838.2021.00010
• 发表时间: 2021
• 期刊: IEEE/ACM Redefining Scalability for Diversely Heterogeneous Architectures Workshop
• 作者: Schmerge, Jordan;Mawhirter, Daniel;Holmes, Connor;McClurg, Jedidiah;Wu, Bo
• 通讯作者: Wu, Bo

FLARE: Flexibly Sharing Commodity GPUs to Enforce QoS and Improve Utilization

• DOI: 10.1007/978-3-030-72789-5_3
• 发表时间: 2019
• 作者: Wei Han;Daniel Mawhirter;Bo Wu;Lin Ma;Chen Tian

AutoMine: harmonizing high-level abstraction and high performance for graph mining

• DOI: 10.1145/3341301.3359633
• 发表时间: 2019-10
• 期刊: Proceedings of the 27th ACM Symposium on Operating Systems Principles
• 作者: Daniel Mawhirter;Bo Wu

GRNN: Low-Latency and Scalable RNN Inference on GPUs

• DOI: 10.1145/3302424.3303949
• 发表时间: 2019-03
• 期刊: Proceedings of the Fourteenth EuroSys Conference 2019
• 作者: Connor Holmes;Daniel Mawhirter;Yuxiong He;Feng Yan;Bo Wu