CRII: CSR: An Asynchronous Design to Reduce the Long Tail Latency of n-Tier Applications in the Cloud

低延迟对于面向Web的电子商务应用程序至关重要。例如，亚马逊报告说，页面加载时间每增加100毫秒，销售额就会下降1%。然而，实践者一直经历云中的web应用的长尾延迟问题或宽响应时间变化，因为系统的利用率达到中等水平（例如，50%）。一个实际的结果是，据报道，企业云数据中心有目的地保持低利用率水平（例如，18%），以避免长尾延迟问题造成的惩罚，浪费了相当多的计算资源和功率。解决延迟长尾问题将对实现云计算资源的更高成本效率产生巨大影响，从而降低云用户的成本和云提供商的回报，以及更可持续的大规模计算基础设施的其他好处。该项目的主要目标是研究和解决导致n层Web应用程序的长尾延迟问题的一个关键因素：跨层队列传播。跨层队列传播是由具有同步层间通信（RPC风格的调用/响应）的n层Web应用程序的请求处理链中的强相互依赖性引起的效果。利用这种跨层队列传播，下游服务器（例如，数据库）可以被传播和放大到上游层（例如，本研究提出一种异步的层间通信设计，以解决n层系统中跨层队列传播引起的长尾延迟问题。异步通信的基本原理是解耦由传统RPC风格的同步通信强制执行的复杂的层间依赖关系。参与异步通信的服务器采用事件驱动的设计，其中一个组件服务器中的处理线程与请求处理链中其他组件服务器中的处理线程“独立”。新的设计有望打破请求处理链中的跨层队列传播，并防止小排队延迟放大为长尾延迟问题。为了验证这一假设，本研究将设计一个具有代表性的异步三层系统，该系统由支持异步通信的组件服务器组成，实现与所开发的异步服务器兼容的异步基准测试应用，并运行大规模的云实验，验证异步设计对打破跨层队列传播的有效性。

Low latency is essential for web facing e-commerce applications. For example, Amazon reported that every 100ms increase in the page load time correlates to a decrease in sales by 1%. However, practitioners have consistently experienced the long tail latency problem, or wide response time variations, of web applications in the cloud as the utilization of the system reaches moderate levels (e.g., 50%). A practical consequence is that enterprise cloud data centers have been reported to purposefully keep utilization levels low (e.g., 18%) to avoid the penalties caused by the long tail latency problem, wasting considerable computing resources and power. Solving the latency long tail problem will have a great impact on achieving higher cost efficiency of cloud computing resources, leading to lower cost of cloud users and higher return of cloud providers, among other benefits of more sustainable large-scale computing infrastructures. The main goal of this project is to investigate and resolve a key factor that causes the long tail latency problem of n-tier web applications: the Cross-Tier Queue Propagation. The Cross-Tier Queue Propagation is an effect caused by strong inter-dependencies in the request processing chain of an n-tier web application with synchronous inter-tier communication (RPC-style call/response). With such Cross-Tier Queue Propagation, a small queuing delay in a downstream server (e.g., a database) can be propagated and amplified to upstream tiers (e.g., a web server), leading to wide response time variations.This research proposes an asynchronous design of inter-tier communication that addresses the long tail latency problem caused by the Cross-Tier Queue Propagation of an n-tier system. The rationale of asynchronous communication is to decouple complex inter-tier dependencies enforced by traditional RPC-style synchronous communication. Servers involved in asynchronous communication adopt an event-driven design in which the processing threads in one component server are "independent" from those in other component servers in the request processing chain. The new design is expected to break the Cross-Tier Queue Propagation in the request processing chain and prevent the amplification of small queuing delay into the long tail latency problem. To validate the hypothesis, this research will design a representative asynchronous 3-tier system consisting of component servers that support asynchronous communication, implement asynchronous benchmark applications that are compatible with the developed asynchronous servers, and run large-scale cloud experiments and validate the effectiveness of the asynchronous design to break the Cross-Tier Queue Propagation.