Sora: A Latency Sensitive Approach for Microservice Soft Resource Adaptation
Sora: A Latency Sensitive Approach for Microservice Soft Resource Adaptation
复制标题
DOI：
10.1145/3590140.3592851
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Jianshu Liu;Qingyang Wang;Shungeng Zhang;Liting Hu;Dilma Da Silva
中科院分区：
文献类型：
--
作者：
Jianshu Liu;Qingyang Wang;Shungeng Zhang;Liting Hu;Dilma Da Silva
由于其业务影响，包括众多分布式和轻量级微服务的现代Web服务的快速响应时间变得越来越重要。虽然仅硬件资源缩放方法（例如，FIRM [47]和PARSLO [40]）已经被提出来减轻关键微服务的响应时间波动，软资源的重新适应（例如，线程或连接），其控制硬件资源使用的并发性。本文表明，关键微服务的软资源自适应对系统可扩展性有重大影响，因为软资源的分配不足或过度都可能导致底层硬件资源的低效使用。我们提出了索拉，一个智能，快速的软资源适配管理框架，用于快速识别和调整关键微服务的最佳并发级别，以减轻服务级别目标（SLO）违规。索拉利用在线细粒度系统指标和沿请求执行的关键路径传播的最后期限沿着，快速准确地为关键微服务提供最佳并发设置。基于六个真实世界的突发工作负载跟踪和两个代表性的微服务基准测试（Sock Shop和Social Network），我们的实验结果表明，与纯硬件扩展策略FIRM [47]相比，索拉可以有效地缓解大的响应时间波动，并将第99百分位延迟减少2.5倍，并将其减少1.5倍到最先进的并发感知系统扩展策略ConScale。
Fast response time for modern web services that include numerous distributed and lightweight microservices becomes increasingly important due to its business impact. While hardware-only resource scaling approaches (e.g., FIRM [47] and PARSLO [40]) have been proposed to mitigate response time fluctuations on critical microservices, the re-adaptation of soft resources (e.g., threads or connections) that control the concurrency of hardware resource usage has been largely ignored. This paper shows that the soft resource adaptation of critical microservices has a significant impact on system scalability because either under- or over-allocation of soft resources can lead to inefficient usage of underlying hardware resources. We present Sora, an intelligent, fast soft resource adaptation management framework for quickly identifying and adjusting the optimal concurrency level of critical microservices to mitigate service-level objective (SLO) violations. Sora leverages online fine-grained system metrics and the propagated deadline along the critical path of request execution to quickly and accurately provide optimal concurrency setting for critical microservices. Based on six real-world bursty workload traces and two representative microservices benchmarks (Sock Shop and Social Network), our experimental results show that Sora can effectively mitigate large response time fluctuations and reduce the 99th percentile latency by up to 2.5× compared to the hardware-only scaling strategy FIRM [47] and 1.5× to the state-of-the-art concurrency-aware system scaling strategy ConScale.