FLEP: Enabling Flexible and Efficient Preemption on GPUs
FLEP: Enabling Flexible and Efficient Preemption on GPUs
复制标题
DOI：
10.1145/3037697.3037742
复制
发表时间：
2017-04
期刊：
影响因子：
--
通讯作者：
Bo Wu;Xu Liu;Xiaobo Zhou;Changjun Jiang
中科院分区：
文献类型：
--
作者：
Bo Wu;Xu Liu;Xiaobo Zhou;Changjun Jiang
gpu被广泛应用于高性能计算和云计算平台，以加速通用工作负载。然而，现代gpu不支持灵活的抢占，导致多任务环境下的性能和优先级反转问题。在本文中，我们提出并开发了FLEP，这是第一个在商用gpu上实现灵活内核抢占和内核调度的软件系统。FLEP编译引擎将GPU程序转换成可抢占的形式，可以在执行过程中中断，并产生GPU中的全部或部分流多处理器（SMs）。FLEP运行时引擎拦截所有内核调用，并确定哪些内核以及应该如何抢占和调度这些内核。双内核协同运行的实验结果表明，高优先级内核的速度提高了24.2倍，具有相同优先级的内核的标准化平均周转时间提高了27倍。当等待内核只需要几个SMs时，与占用整个GPU相比，FLEP减少了高达41%的抢占延迟。尽管有这些优点，但FLEP只引入了2.5%的运行时开销，这比内核切片方法要低得多。
GPUs are widely adopted in HPC and cloud computing platforms to accelerate general-purpose workloads. However, modern GPUs do not support flexible preemption, leading to performance and priority inversion problems in multi-tasking environments. In this paper, we propose and develop FLEP, the first software system that enables flexible kernel preemption and kernel scheduling on commodity GPUs. The FLEP compilation engine transforms the GPU program into preemptable forms, which can be interrupted during execution and yield all or part of the streaming multi-processors (SMs) in the GPU. The FLEP runtime engine intercepts all kernel invocations and determines which kernels and how those kernels should be preempted and scheduled. Experimental results on two-kernel co-runs demonstrate up to 24.2X speedup for high-priority kernels and up to 27X improvement on normalized average turnaround time for kernels with the same priority. FLEP reduces the preemption latency by up to 41% compared to yielding the whole GPU when the waiting kernels only need several SMs. With all the benefits, FLEP only introduces 2.5% runtime overhead, which is substantially lower than the kernel slicing approach.