CRII: CNS: An Experimental Infrastructure to Reduce Latency Long-tail in Real-time Stream Processing

CRII：CNS：减少实时流处理中延迟长尾的实验基础设施

• 批准号: 2245827
• 负责人: Shungeng Zhang
• 金额: $ 17.5万
• 依托单位: AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC.
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2245827&HistoricalAwards=false
• 关键词: CRII; CNS; Experimental; Infrastructure; Reduce

Because of Covid, businesses are reassessing their decades-old traditional strategies. Leading companies such as Walmart are embracing event streaming and cloud-centric web applications to rapidly reshape their business models. However, unexpected performance variances in these web-facing applications could significantly and adversely affect consumers and, ultimately, online businesses. The project addresses this challenge with an experimental infrastructure and research aimed at relieving potential performance concerns from service providers to achieve good performance and high resource efficiency for web-facing applications.Consistent low latency has become essential for user-facing, latency-sensitive web applications such as e-commerce and real-time stream processing due to its business impact. For example, the Bing search engine team reported that a 500-millisecond delay could lead to a decrease in revenue by 1.2%. Despite continuous efforts made by practitioners, the latency long-tail problem still consistently occurs, where a small number of requests take a long response time (e.g., multiple seconds) to return. Existing (and state-of-the-art) infrastructures mainly focus on the correlation between execution time variance localized in the critical paths (i.e., the longest path in time for a request from start to finish) and latency long-tail in web-facing applications. However, our preliminary experimental results suggest that asynchronous, very short but intense resource demands (milliseconds level, referred to as the “millibottlenecks”) outside of critical paths can also cause significant latency long-tail, causing serious end-user dissatisfaction and leading to significant revenues loss. This project targets the unique challenges of millibottlenecks outside of critical paths by designing a fine-grained monitoring toolkit and developing sophisticated quantitative analyses for millibottlenecks as well as effective cures for latency long-tail problems to achieve the goals of reducing latency long-tail in emerging real-time processing applications.This project responds to this research challenge with an experimental infrastructure, to reduce latency long-tail caused by millibottlenecks in real-time stream processing. The project will proceed along with three tasks. First, the experimental infrastructure starts with the initial results of the observed latency long-tail in emerging real-time stream processing through fine-grained performance monitoring (e.g., both system- and application-level metrics). Second, the team will propose to methodically evaluate and validate millibottlenecks (including that outside of the critical path) and to determine their causal relationship to the latency long-tail. Third, effective cures will be proposed to reduce latency long-tail by disrupting the causality from millibottlenecks to the latency long-tail problem with negligible overhead. Furthermore, the knowledge gained on the latency long-tail problem caused by various millibottlenecks will enable more reliable performance studies and long-term advances in the design and implementation of mission-critical web applications in clouds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

由于共同，企业正在重新评估其数十年历史的传统策略。沃尔玛等领先的公司正在拥抱活动流和以云为中心的Web应用程序，以迅速重塑其业务模型。但是，这些面向网络的应用程序中意外的性能差异可能会严重影响消费者，最终影响在线业务。该项目通过实验基础架构和研究来解决这一挑战，旨在缓解服务提供商的潜在绩效问题，以实现良好的性能和针对网络面向应用程序的高资源效率。一致的低潜伏期对于诸如电子商务和实时流动的诸如其业务影响之类的诸如电子商务和实时流媒体上的延迟，对延迟延迟敏感的Web应用程序至关重要。例如，Bing搜索引擎团队报告说，500毫秒延迟可能会导致收入下降1.2％。尽管从业者不断努力，但延迟的长尾问题仍然始终如一地发生，其中少数请求需要很长的响应时间（例如，多秒）才能返回。现有的（和最新的）基础架构主要集中于在关键路径中本地定位的执行时间差异（即，从开始到完成请求的最长途径）与在面向网络应用程序中的延迟延迟长尾。但是，我们的初步实验结果表明，在关键路径之外，异步，非常短但强烈的资源需求（毫秒水平，称为“ Millibottlenecks”）也可能导致重要的潜伏期长尾，从而导致严重的最终用户不满，并导致重大揭示损失。该项目通过设计精细粒度监控工具包并为毫米链勒板进行复杂的定量分析以及有效治愈延迟的长尾问题，以实现延迟效果，从而减少实验性的实验性，该项目针对关键途径以外的毫匹克甲再子的独特挑战。实时流处理中的Millibottlenecks。该项目将与三个任务一起进行。首先，实验基础架构始于通过细粒度的性能监控（例如，系统和应用程序级指标）观察到的潜伏期长尾的初始结果。其次，该团队将提议有条不紊地评估和验证Millibottlenecks（包括在关键路径之外），并确定其与潜伏期长尾的灾难性关系。第三，将提出有效的治疗方法，以减少长尾litch缩，从而破坏从毫米链球链球上的偶然性到可忽略不计的延迟长尾问题。此外，关于各种毫米型造成的潜伏期长尾问题的知识将使更可靠的绩效研究和长期进步在云中的设计和实施至关重要的网络应用程序的长期进步。该奖项反映了NSF的法定任务，并通过使用基金会的知识优点和广泛影响来评估NSF的法定任务，并被认为是宝贵的支持。

项目成果

Sora: A Latency Sensitive Approach for Microservice Soft Resource Adaptation

• DOI: 10.1145/3590140.3592851
• 发表时间: 2023-11
• 期刊: Proceedings of the 24th International Middleware Conference
• 作者: Jianshu Liu;Qingyang Wang;Shungeng Zhang;Liting Hu;Dilma Da Silva