CAREER: Compiler and Runtime Support for Multi-Tasking on Commodity GPUs

通用图形处理器(GPU)计算已经成为主流，在机器学习、图形分析和科学模拟等各个领域都可以看到这一点。一个值得注意的趋势是在数据中心和云计算基础设施中使用GPU，以满足用户日益增长的加速应用程序的需求。在这样的多任务环境中，来自不同用户的应用程序争用共享的GPU，导致不可预测和不可接受的性能下降。这个职业项目旨在开发一套编译器和运行时技术，以透明和高效的方式支持商用GPU上的多任务处理。编译器技术绕过硬件限制以启用一组功能，例如抢占，运行时系统调度应用程序以利用GPU的潜力并保证服务质量。此外，研究人员以一个GPU教育中心为基础，推进了大学的GPU教育，面向计算机科学(CS)和非计算机科学(CS)学生。具体地说，该项目研究了如何通过构建一个实现三个目标的系统来集成编译器和运行时技术来支持GPU上的多任务处理。首先，该系统通过启用灵活的GPU内核抢占来解决GPU核心争用问题。编译器通过绕过硬件线程调度器施加的限制将GPU程序转换为可抢占形式。运行时截取所有GPU内核启动请求，并做出全局抢占和调度决策，以最大化性能。其次，该系统支持不同应用程序的线程细粒度共享，充分利用GPU流媒体多处理器内的硬件资源。运行时保证了面向用户的应用程序的服务质量，同时借助性能预测优化了总体吞吐量。第三，该系统通过协调GPU内存传输来解决GPU内存争用问题，其中考虑了内存访问模式和数组重用模式。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

General-purpose Graphics Processing Units (GPU) computing has become mainstream, as witnessed in various domains such as machine learning, graph analytics, and scientific simulation. One notable trend is employing GPUs in data centers and cloud computing infrastructures to satisfy users' increasing demand to accelerate their applications. In such multi-tasking environments, applications from different users contend to use the shared GPU, leading to unpredictable and unacceptable performance degradation. This CAREER project aims at developing a set of compiler and runtime techniques to support multi-tasking on commodity GPUs in a transparent and efficient manner. The compiler techniques circumvent the hardware limitations to enable a set of features, such as preemption, and the runtime system schedules applications to utilize the potential of the GPU and guarantees quality of service. In addition, the investigator advances GPU education in the University to target both Computer Science (CS) and non-CS students based on a GPU education center.Specifically, the project investigates how to integrate compiler and runtime techniques to support multi-tasking on GPUs by building a system that achieves three goals. First, the system addresses GPU core contention by enabling flexible GPU kernel preemption. The compiler transforms the GPU program to be a preemptable form by circumventing the limitation imposed by the hardware thread scheduler. The runtime intercepts all GPU kernel launch requests and makes global preemption and scheduling decisions to maximize performance. Second, the system supports fine-grained sharing for threads from different applications to fully utilize hardware resources within GPU streaming multi-processors. The runtime guarantees the QoS of user-facing applications while optimizing overall throughput aided by performance prediction. Third, the system addresses GPU memory contention by coordinating GPU memory transfers, which considers memory access patterns and array reuse patterns.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.