CRII: CNS: An Experimental Infrastructure to Reduce Latency Long-tail in Real-time Stream Processing

CRII：CNS：减少实时流处理中延迟长尾的实验基础设施

• 批准号: 2245827
• 负责人: Shungeng Zhang
• 金额: $ 17.5万
• 依托单位: AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC.
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2245827&HistoricalAwards=false
• 关键词: CRII; CNS; Experimental; Infrastructure; Reduce

Because of Covid, businesses are reassessing their decades-old traditional strategies. Leading companies such as Walmart are embracing event streaming and cloud-centric web applications to rapidly reshape their business models. However, unexpected performance variances in these web-facing applications could significantly and adversely affect consumers and, ultimately, online businesses. The project addresses this challenge with an experimental infrastructure and research aimed at relieving potential performance concerns from service providers to achieve good performance and high resource efficiency for web-facing applications.Consistent low latency has become essential for user-facing, latency-sensitive web applications such as e-commerce and real-time stream processing due to its business impact. For example, the Bing search engine team reported that a 500-millisecond delay could lead to a decrease in revenue by 1.2%. Despite continuous efforts made by practitioners, the latency long-tail problem still consistently occurs, where a small number of requests take a long response time (e.g., multiple seconds) to return. Existing (and state-of-the-art) infrastructures mainly focus on the correlation between execution time variance localized in the critical paths (i.e., the longest path in time for a request from start to finish) and latency long-tail in web-facing applications. However, our preliminary experimental results suggest that asynchronous, very short but intense resource demands (milliseconds level, referred to as the “millibottlenecks”) outside of critical paths can also cause significant latency long-tail, causing serious end-user dissatisfaction and leading to significant revenues loss. This project targets the unique challenges of millibottlenecks outside of critical paths by designing a fine-grained monitoring toolkit and developing sophisticated quantitative analyses for millibottlenecks as well as effective cures for latency long-tail problems to achieve the goals of reducing latency long-tail in emerging real-time processing applications.This project responds to this research challenge with an experimental infrastructure, to reduce latency long-tail caused by millibottlenecks in real-time stream processing. The project will proceed along with three tasks. First, the experimental infrastructure starts with the initial results of the observed latency long-tail in emerging real-time stream processing through fine-grained performance monitoring (e.g., both system- and application-level metrics). Second, the team will propose to methodically evaluate and validate millibottlenecks (including that outside of the critical path) and to determine their causal relationship to the latency long-tail. Third, effective cures will be proposed to reduce latency long-tail by disrupting the causality from millibottlenecks to the latency long-tail problem with negligible overhead. Furthermore, the knowledge gained on the latency long-tail problem caused by various millibottlenecks will enable more reliable performance studies and long-term advances in the design and implementation of mission-critical web applications in clouds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

由于新冠肺炎疫情，企业正在重新评估其数十年的传统战略。 像沃尔玛这样的领先公司正在拥抱事件流和以云为中心的Web应用程序，以快速重塑其商业模式。然而，这些面向Web的应用程序中的意外性能差异可能会对消费者产生重大不利影响，并最终影响在线业务。该项目通过实验性的基础设施和研究来应对这一挑战，旨在缓解服务提供商对性能的潜在担忧，从而为面向Web的应用程序实现良好的性能和高资源效率。一致的低延迟对于面向用户的、对延迟敏感的Web应用程序（如电子商务和实时流处理）来说至关重要，因为其业务影响。例如，必应搜索引擎团队报告称，500毫秒的延迟可能导致收入减少1.2%。尽管从业者做出了持续的努力，但延迟长尾问题仍然持续发生，其中少量的请求需要很长的响应时间（例如，几秒钟）返回。现有的（和最先进的）基础设施主要集中在关键路径中局部化的执行时间方差（即，请求从开始到结束的最长时间路径）和面向web的应用程序中的延迟长尾。然而，我们的初步实验结果表明，关键路径之外的异步、非常短但密集的资源需求（毫秒级，称为“毫瓶颈”）也可能导致显著的延迟长尾，导致严重的最终用户不满，并导致重大的收入损失。该项目针对关键路径之外的毫瓶颈的独特挑战，通过设计细粒度的监控工具包，开发针对毫瓶颈的复杂定量分析以及针对延迟长尾问题的有效解决方案，以实现在新兴的实时处理应用中减少延迟长尾的目标。该项目以实验基础设施响应这一研究挑战，以减少实时流处理中由毫瓶颈引起的延迟长尾。该项目将沿着三项任务进行。首先，实验基础设施通过细粒度性能监控（例如，系统级和应用级度量）。其次，团队将建议系统地评估和验证毫瓶颈（包括关键路径之外的毫瓶颈），并确定它们与延迟长尾的因果关系。第三，将提出有效的治疗，以减少延迟长尾破坏的因果关系，从毫瓶颈的延迟长尾问题，可以忽略不计的开销。此外，在各种毫瓶颈造成的延迟长尾问题上获得的知识将使云计算中任务关键型Web应用程序的设计和实施更可靠的性能研究和长期进步成为可能。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Sora: A Latency Sensitive Approach for Microservice Soft Resource Adaptation

• DOI: 10.1145/3590140.3592851
• 发表时间: 2023-11
• 期刊: Proceedings of the 24th International Middleware Conference
• 作者: Jianshu Liu;Qingyang Wang;Shungeng Zhang;Liting Hu;Dilma Da Silva