CRII: CSR: An Asynchronous Design to Reduce the Long Tail Latency of n-Tier Applications in the Cloud

低延迟对于面向web的电子商务应用程序至关重要。例如，亚马逊报告说，页面加载时间每增加100毫秒，销售额就会下降1%。然而，当系统的利用率达到中等水平（例如，50%）时，从业者一直会遇到云中的web应用程序的长尾延迟问题，或者响应时间变化很大。一个实际的后果是，据报道，企业云数据中心有意保持较低的利用率水平（例如18%），以避免长尾延迟问题造成的损失，浪费大量的计算资源和电力。解决延迟长尾问题将对实现更高的云计算资源成本效率产生重大影响，从而降低云用户的成本，提高云提供商的回报，以及更可持续的大规模计算基础设施带来的其他好处。该项目的主要目标是研究和解决导致n层web应用程序长尾延迟问题的关键因素：跨层队列传播。跨层队列传播是由n层web应用程序的请求处理链中具有同步层间通信（rpc风格的调用/响应）的强相互依赖引起的效应。使用这种跨层队列传播，下游服务器（例如，数据库）中的小队列延迟可以传播并放大到上游层（例如，web服务器），从而导致大的响应时间变化。本研究提出一种层间通信的异步设计，以解决n层系统的跨层队列传播所造成的长尾延迟问题。异步通信的基本原理是解耦由传统rpc风格的同步通信强制执行的复杂层间依赖。涉及异步通信的服务器采用事件驱动设计，其中一个组件服务器中的处理线程与请求处理链中其他组件服务器中的处理线程“独立”。新设计有望打破请求处理链中的跨层队列传播，防止小队列延迟放大为长尾延迟问题。为了验证这一假设，本研究将设计一个具有代表性的异步三层系统，由支持异步通信的组件服务器组成，实现与所开发的异步服务器兼容的异步基准应用程序，并进行大规模云实验，验证异步设计对打破跨层队列传播的有效性。

Low latency is essential for web facing e-commerce applications. For example, Amazon reported that every 100ms increase in the page load time correlates to a decrease in sales by 1%. However, practitioners have consistently experienced the long tail latency problem, or wide response time variations, of web applications in the cloud as the utilization of the system reaches moderate levels (e.g., 50%). A practical consequence is that enterprise cloud data centers have been reported to purposefully keep utilization levels low (e.g., 18%) to avoid the penalties caused by the long tail latency problem, wasting considerable computing resources and power. Solving the latency long tail problem will have a great impact on achieving higher cost efficiency of cloud computing resources, leading to lower cost of cloud users and higher return of cloud providers, among other benefits of more sustainable large-scale computing infrastructures. The main goal of this project is to investigate and resolve a key factor that causes the long tail latency problem of n-tier web applications: the Cross-Tier Queue Propagation. The Cross-Tier Queue Propagation is an effect caused by strong inter-dependencies in the request processing chain of an n-tier web application with synchronous inter-tier communication (RPC-style call/response). With such Cross-Tier Queue Propagation, a small queuing delay in a downstream server (e.g., a database) can be propagated and amplified to upstream tiers (e.g., a web server), leading to wide response time variations.This research proposes an asynchronous design of inter-tier communication that addresses the long tail latency problem caused by the Cross-Tier Queue Propagation of an n-tier system. The rationale of asynchronous communication is to decouple complex inter-tier dependencies enforced by traditional RPC-style synchronous communication. Servers involved in asynchronous communication adopt an event-driven design in which the processing threads in one component server are "independent" from those in other component servers in the request processing chain. The new design is expected to break the Cross-Tier Queue Propagation in the request processing chain and prevent the amplification of small queuing delay into the long tail latency problem. To validate the hypothesis, this research will design a representative asynchronous 3-tier system consisting of component servers that support asynchronous communication, implement asynchronous benchmark applications that are compatible with the developed asynchronous servers, and run large-scale cloud experiments and validate the effectiveness of the asynchronous design to break the Cross-Tier Queue Propagation.