CRII: CNS: An Experimental Infrastructure to Reduce Latency Long-tail in Real-time Stream Processing

由于新冠疫情，企业正在重新评估其数十年的传统战略。沃尔玛（Walmart）等领先公司正在采用事件流媒体和以云为中心的web应用程序来快速重塑其业务模式。然而，在这些面向web的应用程序中，意想不到的性能差异可能会对消费者产生重大的不利影响，并最终影响在线业务。该项目通过实验性基础设施和研究来解决这一挑战，旨在缓解服务提供商对潜在性能的担忧，从而为面向web的应用程序实现良好的性能和高资源效率。由于其业务影响，一致的低延迟对于面向用户、对延迟敏感的web应用程序（如电子商务和实时流处理）已经变得至关重要。例如，必应搜索引擎团队报告说，500毫秒的延迟可能导致收入下降1.2%。尽管从业者做出了持续的努力，延迟长尾问题仍然持续发生，其中少量请求需要很长的响应时间（例如，数秒）才能返回。现有的（和最先进的）基础设施主要关注关键路径（即请求从开始到结束的最长时间路径）的执行时间差异与面向web的应用程序中的延迟长尾之间的相关性。然而，我们的初步实验结果表明，关键路径之外的异步、非常短但强烈的资源需求（毫秒级，称为“毫瓶颈”）也可能导致严重的延迟长尾，导致严重的最终用户不满并导致重大的收入损失。该项目针对关键路径之外的微瓶颈的独特挑战，设计了一个细粒度监控工具包，开发了针对微瓶颈的复杂定量分析，以及针对延迟长尾问题的有效解决方案，以实现在新兴实时处理应用程序中减少延迟长尾的目标。该项目通过实验基础设施来应对这一研究挑战，以减少实时流处理中由微瓶颈引起的延迟长尾。该项目将与三个任务一起进行。首先，实验基础设施通过细粒度的性能监控（例如，系统级和应用级指标），从观察到的实时流处理中的延迟长尾的初始结果开始。其次，团队将建议系统地评估和验证微瓶颈（包括关键路径之外的微瓶颈），并确定它们与延迟长尾的因果关系。第三，将提出有效的解决方案，通过破坏从微瓶颈到延迟长尾问题的因果关系，以忽略开销来减少延迟长尾问题。此外，在各种微瓶颈引起的延迟长尾问题上获得的知识将使云中的关键任务web应用程序的设计和实现更加可靠的性能研究和长期进展成为可能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Because of Covid, businesses are reassessing their decades-old traditional strategies. Leading companies such as Walmart are embracing event streaming and cloud-centric web applications to rapidly reshape their business models. However, unexpected performance variances in these web-facing applications could significantly and adversely affect consumers and, ultimately, online businesses. The project addresses this challenge with an experimental infrastructure and research aimed at relieving potential performance concerns from service providers to achieve good performance and high resource efficiency for web-facing applications.Consistent low latency has become essential for user-facing, latency-sensitive web applications such as e-commerce and real-time stream processing due to its business impact. For example, the Bing search engine team reported that a 500-millisecond delay could lead to a decrease in revenue by 1.2%. Despite continuous efforts made by practitioners, the latency long-tail problem still consistently occurs, where a small number of requests take a long response time (e.g., multiple seconds) to return. Existing (and state-of-the-art) infrastructures mainly focus on the correlation between execution time variance localized in the critical paths (i.e., the longest path in time for a request from start to finish) and latency long-tail in web-facing applications. However, our preliminary experimental results suggest that asynchronous, very short but intense resource demands (milliseconds level, referred to as the “millibottlenecks”) outside of critical paths can also cause significant latency long-tail, causing serious end-user dissatisfaction and leading to significant revenues loss. This project targets the unique challenges of millibottlenecks outside of critical paths by designing a fine-grained monitoring toolkit and developing sophisticated quantitative analyses for millibottlenecks as well as effective cures for latency long-tail problems to achieve the goals of reducing latency long-tail in emerging real-time processing applications.This project responds to this research challenge with an experimental infrastructure, to reduce latency long-tail caused by millibottlenecks in real-time stream processing. The project will proceed along with three tasks. First, the experimental infrastructure starts with the initial results of the observed latency long-tail in emerging real-time stream processing through fine-grained performance monitoring (e.g., both system- and application-level metrics). Second, the team will propose to methodically evaluate and validate millibottlenecks (including that outside of the critical path) and to determine their causal relationship to the latency long-tail. Third, effective cures will be proposed to reduce latency long-tail by disrupting the causality from millibottlenecks to the latency long-tail problem with negligible overhead. Furthermore, the knowledge gained on the latency long-tail problem caused by various millibottlenecks will enable more reliable performance studies and long-term advances in the design and implementation of mission-critical web applications in clouds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.