Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling
Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling
复制标题
DOI：
10.1109/tpds.2013.257
复制
发表时间：
2013-03
影响因子：
5.3
通讯作者：
Jianlong Zhong;Bingsheng He
中科院分区：
文献类型：
--
作者：
Jianlong Zhong;Bingsheng He
图形处理器（gpu）最近被广泛用作集群和云等共享环境中的加速器。在这样的共享环境中，许多内核由不同的用户提交给gpu，吞吐量是性能和总拥有成本的重要指标。尽管最近改进了对并发GPU内核执行的运行时支持，但GPU可能严重未被充分利用，导致吞吐量次优。在本文中，我们提出了Kernelet，一个运行时系统，以提高GPU上并发内核执行的吞吐量。Kernelet包含透明内存管理和PCI-e数据传输技术，以及用于内核执行的动态切片和调度技术。通过切片，Kernelet将GPU内核划分为多个子内核（即切片）。每个片都有可调的占用，以允许与其他片共同调度以获得高GPU利用率。提出了一种新的基于马尔可夫链的性能模型来指导调度决策。我们的实验结果表明，在NVIDIA Tesla C2050和GTX680 gpu上，性能分别提高了31%和23%。
Graphics processors, or GPUs, have recently been widely used as accelerators in shared environments such as clusters and clouds. In such shared environments, many kernels are submitted to GPUs from different users, and throughput is an important metric for performance and total ownership cost. Despite recently improved runtime support for concurrent GPU kernel executions, the GPU can be severely underutilized, resulting in suboptimal throughput. In this paper, we propose Kernelet, a runtime system to improve the throughput of concurrent kernel executions on the GPU. Kernelet embraces transparent memory management and PCI-e data transfer techniques, and dynamic slicing and scheduling techniques for kernel executions. With slicing, Kernelet divides a GPU kernel into multiple sub-kernels (namely slices ). Each slice has tunable occupancy to allow co-scheduling with other slices for high GPU utilization. We develop a novel Markov chain-based performance model to guide the scheduling decision. Our experimental results demonstrate up to 31 percent and 23 percent performance improvement on NVIDIA Tesla C2050 and GTX680 GPUs, respectively.