Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution
Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution
复制标题
DOI：
10.1145/3326124
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Zhen Lin;Hongwen Dai;Mike Mantor;Huiyang Zhou
中科院分区：
文献类型：
--
作者：
Zhen Lin;Hongwen Dai;Mike Mantor;Huiyang Zhou
当代GPU支持多个内核在相同的流式多处理器（SM）上并发运行。最近的研究表明，这种并发内核执行（CKE）提高了资源利用率和计算吞吐量。现有的研究主要集中在协同线程阵列（CTA）或线程调度器（warp scheduler）级别上对GPU资源进行分区，以提高协同执行效率。然而，显着的性能放缓和不公平的观察时，延迟敏感的内核与带宽密集型的共同运行。原因在于，来自带宽密集型内核的带宽过度订阅导致了更严重的存储器访问延迟，这对延迟敏感的内核是非常有害的。即使在带宽密集型内核中，更密集的内核也可能比不太密集的内核不公平地消耗更高的带宽。在本文中，我们首先提出了一个案例，这些问题不能充分解决的CTA组合单独管理，并揭示了根本原因。然后，我们提出了一个协调的方法CTA组合和带宽划分。我们的方法动态检测共同运行的内核延迟敏感或带宽密集。由于DRAM带宽和L2到L1片上网络（NoC）带宽都可能是关键资源，因此我们的方法沿着选择适当的CTA组合来协调地划分带宽资源。关键目标是为延迟敏感的内核分配更多的CTA资源，为NoC/DRAM密集型内核分配更多的NoC/DRAM带宽资源。我们使用主导资源公平性（DRF）的变体来实现它。与两种最先进的CKE优化方案SMK [52]和WS [55]相比，我们的方法分别将平均谐波加速比提高了78%和39%。即使与从所有可能的CTA组合中进行穷举搜索获得的最佳CTA组合相比，我们的方法平均将谐波加速比提高了51%和11%。
Contemporary GPUs support multiple kernels to run concurrently on the same streaming multiprocessors (SMs). Recent studies have demonstrated that such concurrent kernel execution (CKE) improves both resource utilization and computational throughput. Most of the prior works focus on partitioning the GPU resources at the cooperative thread array (CTA) level or the warp scheduler level to improve CKE. However, significant performance slowdown and unfairness are observed when latency-sensitive kernels co-run with bandwidth-intensive ones. The reason is that bandwidth over-subscription from bandwidth-intensive kernels leads to much aggravated memory access latency, which is highly detrimental to latency-sensitive kernels. Even among bandwidth-intensive kernels, more intensive kernels may unfairly consume much higher bandwidth than less-intensive ones. In this article, we first make a case that such problems cannot be sufficiently solved by managing CTA combinations alone and reveal the fundamental reasons. Then, we propose a coordinated approach for CTA combination and bandwidth partitioning. Our approach dynamically detects co-running kernels as latency sensitive or bandwidth intensive. As both the DRAM bandwidth and L2-to-L1 Network-on-Chip (NoC) bandwidth can be the critical resource, our approach partitions both bandwidth resources coordinately along with selecting proper CTA combinations. The key objective is to allocate more CTA resources for latency-sensitive kernels and more NoC/DRAM bandwidth resources to NoC-/DRAM-intensive kernels. We achieve it using a variation of dominant resource fairness (DRF). Compared with two state-of-the-art CKE optimization schemes, SMK [52] and WS [55], our approach improves the average harmonic speedup by 78% and 39%, respectively. Even compared to the best possible CTA combinations, which are obtained from an exhaustive search among all possible CTA combinations, our approach improves the harmonic speedup by up to 51% and 11% on average.