CSR: Small: Lightning in Clouds: Detection and Characterization of Very Short Bottlenecks

对于数据中心的持续低利用率（Gartner报告约为18%），一个合理的解释是，管理人员需要保持服务质量，以应对众所周知的延迟长尾问题，即一些通常在几毫秒内返回的看似随机的请求突然需要几秒钟。延迟长尾问题出现在中等利用率水平（例如，50%），所有资源远未饱和的情况下。尽管人们努力以各种方式解决延迟长尾问题，但其原因仍然难以捉摸：在大多数情况下，花费几秒钟的请求在自己执行时实际上在几毫秒内返回。研究和解决延迟长尾问题将有助于在保持服务质量的同时更好地利用，从而降低云用户的成本，提高云提供商的投资回报，降低环境的功耗。这个项目的主要目标是研究一类非常短的瓶颈，在这种瓶颈中，CPU只在不到一秒的时间内饱和，这是延迟长尾问题的一个重要原因。尽管它们的生命周期很短，但由于在请求处理期间各层之间存在很强的依赖性，非常短的瓶颈会在n层应用程序系统的请求链中上下传播排队效应，从而导致显著的响应时间增加（几秒钟）。该项目在云和模拟器中进行大规模实验，以在非常短的瓶颈调查中生成广泛的细粒度监控数据，这些瓶颈在典型的性能监控工具下几乎是不可见的，采样周期为几秒或几分钟。为了匹配非常短的瓶颈的时间尺度，正在改进特殊的仪器软件工具，以毫秒分辨率采样服务器内部的资源利用率，并以微秒分辨率采样服务器间的时间戳消息。对具有自然突发工作负载的n层应用程序基准测试的初步研究发现，在几个系统层（系统软件（JVM垃圾收集）、处理器架构（动态电压和频率缩放）以及虚拟化云环境中的应用程序整合）中，非常短的瓶颈会导致延迟长尾。它们显示了许多其他非常短的瓶颈来源的潜在可能性，例如，内核守护进程在几毫秒内使用100%的CPU。通过对实验数据进行仔细的分布式事件分析，可以发现、验证、再现和详细研究新的极短瓶颈。针对特定的非常短的瓶颈，已经开发了具体的解决方案，例如，改进的Java垃圾收集器。然而，其他非常短的瓶颈没有特定的错误修复，例如，那些由合并的工作负载重叠统计性质的突发事件造成的瓶颈。作为错误修复的替代方案，人们正在探索破坏排队传播的更通用的解决方案。作为一个具体的例子，与使用经典的请求/响应方法（等待线程参与队列传播）不同，正在研究使用带有响应通知的异步请求来减少总体队列，以作为消除或减少几种非常短的瓶颈影响的潜在解决方案。

A plausible explanation for the persistent low utilization of data centers (around 18% by Gartner reports) is the managerial need to maintain quality of service against the well-known Latency Long Tail problem, where some apparently random requests that normally return within milliseconds would suddenly take multiple seconds. The latency long tail problem arises at moderate utilization levels (e.g., 50%) with all resources far from saturation. Despite the efforts to remedy the latency long tail problem in various ways, its causes have remained elusive: In most cases, the very requests that took several seconds actually return within milliseconds when executed by themselves. Studying and solving the latency long tail problem will contribute to better utilization while maintaining quality of service, leading to lower costs for cloud users, higher return on investment for cloud providers, and lower power consumption for the environment. The main goal of this project is the investigation of the class of very short bottlenecks, in which the CPU becomes saturated only for a small fraction of a second, as a significant cause of latency long tail problems. Despite their short lifespan, very short bottlenecks can lead to significant response time increases (several seconds) by propagating queuing effects up and down the request chain in an n-tier application system because of strong dependencies among the tiers during request processing. This project runs large scale experiments in clouds and simulators to generate extensive fine-grain monitoring data in the investigation of very short bottlenecks, which are virtually invisible under typical performance monitoring tools with sampling periods of seconds or minutes. To match the time scale of very short bottlenecks, special instrumentation software tools are being refined to sample intra-server resource utilization at millisecond resolution and timestamp inter-server messages at microsecond resolution. Preliminary studies of n-tier application benchmarks with naturally bursty workloads have found very short bottlenecks that cause latency long tail in several system layers: systems software (JVM garbage collection), processor architecture (dynamic voltage and frequency scaling), and consolidation of applications in virtualized cloud environments. They show the potential for many other sources of very short bottlenecks, e.g., kernel daemon processes that use 100% of CPU for several milliseconds. Through careful distributed event analysis of the experimental data, new kinds of very short bottlenecks can be discovered, verified, reproduced, and studied in detail. Concrete solutions for specific very short bottlenecks have been developed, e.g., an improved Java garbage collector. However, other very short bottlenecks have no specific bug-fixes, e.g., those created by consolidated workload overlapping bursts of statistical nature. As an alternative to bug-fixes, more general solutions that disrupt queuing propagation are being explored. As a concrete example, instead of using a classic request/response approach, where waiting threads participate in the queuing propagation, asynchronous requests with notification of responses to reduce overall queuing is being investigated as a potential solution to eliminate or reduce the impact of several kinds of very short bottlenecks.