CSR: Small: Lightning in Clouds: Detection and Characterization of Very Short Bottlenecks

对于数据中心持续的低利用率（Gartner报告约为18%），一个合理的解释是管理层需要针对众所周知的延迟长尾问题保持服务质量，其中一些通常在毫秒内返回的明显随机请求会突然花费数秒。延迟长尾问题在中等利用水平（例如，50%），所有资源远未饱和。尽管人们努力以各种方式解决延迟长尾问题，但其原因仍然难以捉摸：在大多数情况下，花费几秒钟的请求实际上在执行时会在几毫秒内返回。研究和解决延迟长尾问题将有助于提高利用率，同时保持服务质量，从而降低云用户的成本，提高云提供商的投资回报率，并降低环境的功耗。该项目的主要目标是调查一类非常短的瓶颈，其中CPU仅在一小部分秒内饱和，这是延迟长尾问题的重要原因。尽管它们的生命周期很短，但由于在请求处理期间层之间的强依赖性，非常短的瓶颈可以通过在n层应用程序系统中的请求链中向上和向下传播排队效应来导致显著的响应时间增加（几秒）。该项目在云和模拟器中运行大规模实验，以生成大量细粒度的监控数据，用于调查非常短的瓶颈，这些瓶颈在典型的性能监控工具下几乎不可见，采样周期为秒或分钟。为了与非常短的瓶颈的时间尺度相匹配，正在改进特殊的检测软件工具，以毫秒分辨率对服务器内资源利用率进行采样，并以微秒分辨率对服务器间消息进行时间戳。对具有自然突发工作负载的n层应用程序基准测试的初步研究发现，非常短的瓶颈会在几个系统层中导致延迟长尾：系统软件（JVM垃圾收集），处理器架构（动态电压和频率缩放）以及虚拟化云环境中的应用程序整合。它们显示了许多其他来源的非常短的瓶颈的潜力，例如，内核守护进程在几毫秒内使用100%的CPU。通过对实验数据进行仔细的分布式事件分析，可以发现、验证、重现和详细研究新的非常短的瓶颈。针对特定的极短瓶颈已经开发出了具体的解决方案，例如，一个改进的Java垃圾收集器。然而，其他非常短的瓶颈没有特定的错误修复，例如，这些问题是由合并的工作量重叠统计性质的突发造成的。作为错误修复的替代方案，正在探索更通用的中断队列传播的解决方案。作为一个具体的例子，而不是使用一个经典的请求/响应的方法，其中等待线程参与排队传播，异步请求与响应通知，以减少整体排队正在研究作为一个潜在的解决方案，以消除或减少几种非常短的瓶颈的影响。

A plausible explanation for the persistent low utilization of data centers (around 18% by Gartner reports) is the managerial need to maintain quality of service against the well-known Latency Long Tail problem, where some apparently random requests that normally return within milliseconds would suddenly take multiple seconds. The latency long tail problem arises at moderate utilization levels (e.g., 50%) with all resources far from saturation. Despite the efforts to remedy the latency long tail problem in various ways, its causes have remained elusive: In most cases, the very requests that took several seconds actually return within milliseconds when executed by themselves. Studying and solving the latency long tail problem will contribute to better utilization while maintaining quality of service, leading to lower costs for cloud users, higher return on investment for cloud providers, and lower power consumption for the environment. The main goal of this project is the investigation of the class of very short bottlenecks, in which the CPU becomes saturated only for a small fraction of a second, as a significant cause of latency long tail problems. Despite their short lifespan, very short bottlenecks can lead to significant response time increases (several seconds) by propagating queuing effects up and down the request chain in an n-tier application system because of strong dependencies among the tiers during request processing. This project runs large scale experiments in clouds and simulators to generate extensive fine-grain monitoring data in the investigation of very short bottlenecks, which are virtually invisible under typical performance monitoring tools with sampling periods of seconds or minutes. To match the time scale of very short bottlenecks, special instrumentation software tools are being refined to sample intra-server resource utilization at millisecond resolution and timestamp inter-server messages at microsecond resolution. Preliminary studies of n-tier application benchmarks with naturally bursty workloads have found very short bottlenecks that cause latency long tail in several system layers: systems software (JVM garbage collection), processor architecture (dynamic voltage and frequency scaling), and consolidation of applications in virtualized cloud environments. They show the potential for many other sources of very short bottlenecks, e.g., kernel daemon processes that use 100% of CPU for several milliseconds. Through careful distributed event analysis of the experimental data, new kinds of very short bottlenecks can be discovered, verified, reproduced, and studied in detail. Concrete solutions for specific very short bottlenecks have been developed, e.g., an improved Java garbage collector. However, other very short bottlenecks have no specific bug-fixes, e.g., those created by consolidated workload overlapping bursts of statistical nature. As an alternative to bug-fixes, more general solutions that disrupt queuing propagation are being explored. As a concrete example, instead of using a classic request/response approach, where waiting threads participate in the queuing propagation, asynchronous requests with notification of responses to reduce overall queuing is being investigated as a potential solution to eliminate or reduce the impact of several kinds of very short bottlenecks.