TailGuard: Tail Latency SLO Guaranteed Task Scheduling for Data-Intensive User-Facing Applications
TailGuard: Tail Latency SLO Guaranteed Task Scheduling for Data-Intensive User-Facing Applications
复制标题
DOI：
10.1109/icdcs57875.2023.00042
复制
发表时间：
2023-07
期刊：
影响因子：
--
通讯作者：
Zhijun Wang;Huiyang Li;Lin Sun;Todd Rosenkrantz;Hao Che;Hong Jiang
中科院分区：
文献类型：
--
作者：
Zhijun Wang;Huiyang Li;Lin Sun;Todd Rosenkrantz;Hao Che;Hong Jiang
用于云和边缘计算的数据密集型面向用户（DU）服务的主要设计目标是最大化查询吞吐量，同时满足单个查询的查询尾延迟服务水平目标（SLO）。不幸的是，现有的解决方案达不到这一设计目标，我们认为，这在很大程度上是由于这样一个事实，即他们没有考虑到查询扇出明确。在本文中，我们提出了TailGuard的基础上的尾部延迟SLO和扇出感知最早截止日期第一优先策略（TF-EDFQ）的任务排队在各个任务服务器的查询任务扇出。由于每个任务的任务排队截止时间是基于查询尾延迟SLO和查询扇出得出的，TailGuard为实现设计目标迈出了重要的第一步。通过仿真对TailGuard进行了先进先出（FIFO）任务排队、任务优先级优先级排序（PRIQ）和尾部延迟SLO感知EDFQ（T-EDFQ）策略的评估。它由Tailbench基准测试套件中的三种类型的应用程序驱动。结果表明，与其他三种策略相比，TailGuard可以将资源利用率提高高达80%，同时满足目标尾部延迟SLO。TailGuard还在一个高度异构的数据传感服务的传感即服务（SaS）测试平台中进行了实施和测试，测试结果与其他测试结果一致。
A primary design objective for Data-intensive User-facing (DU) services for cloud and edge computing is to maximize query throughput, while meeting query tail latency Service Level Objectives (SLOs) for individual queries. Unfortunately, the existing solutions fall short of achieving this design objective, which we argue, is largely attributed to the fact that they fail to take the query fanout explicitly into account. In this paper, we propose TailGuard based on a Tail-latency-SLO-and-Fanout-aware Earliest-Deadline-First Queuing policy (TF-EDFQ) for task queuing at individual task servers the query tasks are fanned out to. With the task queuing deadline for each task being derived based on both query tail latency SLO and query fanout, TailGuard takes an important first step towards achieving the design objective. TailGuard is evaluated against First-In-First-Out (FIFO) task queuing, task PRIority Queuing (PRIQ) and Tail-latency-SLO-aware EDFQ (T-EDFQ) policies by simulation. It is driven by three types of applications in the Tailbench benchmark suite. The results demonstrate that TailGuard can improve resource utilization by up to 80%, while meeting the targeted tail latency SLOs, as compared with the other three policies. TailGuard is also implemented and tested in a highly heterogeneous Sensing-as-a-Service (SaS) testbed for a data sensing service, with test results in line with the other ones.