FLARE: Flexibly Sharing Commodity GPUs to Enforce QoS and Improve Utilization
FLARE: Flexibly Sharing Commodity GPUs to Enforce QoS and Improve Utilization
复制标题
DOI：
10.1007/978-3-030-72789-5_3
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Wei Han;Daniel Mawhirter;Bo Wu;Lin Ma;Chen Tian
中科院分区：
文献类型：
--
作者：
Wei Han;Daniel Mawhirter;Bo Wu;Lin Ma;Chen Tian
现代GPU在芯片上集成了数十个流式多处理器（SM）。当在数据中心中使用时，GPU经常遭受独占访问预留的利用不足，因此需要多任务处理（即，共同运行的应用程序），以降低总拥有成本。然而，延迟关键型应用程序可能会遇到太多干扰，无法满足服务质量（QoS）目标。在本文中，我们提出了一个软件系统，FLARE，在空间上共享商品GPU之间的延迟关键型应用程序和尽力而为的应用程序，以执行QoS，以及最大限度地提高整体吞吐量。通过转换尽力而为型应用程序的内核，FLARE在SM中为共同运行的应用程序启用SM分区和线程块分区。它使用微基准引导的静态配置搜索与在线动态搜索相结合，以定位最优（接近最优）的资源分区策略。通过对11个基准测试和2个实际应用程序的评估，FLARE将硬件利用率平均提高了1.39倍。
A modern GPU integrates tens of streaming multi-processors (SMs) on the chip. When used in data centers, the GPUs often suffer from under-utilization for exclusive access reservations, hence demanding multitasking (i.e., co-running applications) to reduce the total cost of ownership. However, latency-critical applications may experience too much interference to meet Quality-of-Service (QoS) targets. In this paper, we propose a software system, FLARE, to spatially share commodity GPUs between latency-critical applications and best-effort applications to enforce QoS as well as maximize overall throughput. By transforming the kernels of best-effort applications, FLARE enables both SM partitioning and thread block partitioning within an SM for co-running applications. It uses a microbenchmark guided static configuration search combined with online dynamic search to locate the optimal (near-optimal) strategy to partition resources. Evaluated on 11 benchmarks and 2 real-world applications, FLARE improves hardware utilization by an average of 1.39X compared to the preemption-based approach.