CSR: Small: Telescopic Analysis for Black-Box Troubleshooting of Distributed Systems

该项目正在开发可扩展的机制，以调试、监控和评估代表现代软件基础设施主干的复杂分布式系统的质量。 这些方法基本上是高度自动化的;它们对分布式系统的操作进行推理，同时将这些系统的组件视为黑盒。 这意味着这些方法不需要源代码、程序员注释或开发人员输入来对分布式系统进行故障排除。 相反，它们依赖于从预先存在的日志消息中收集的详细信息，这些日志消息在非常大规模的分布式系统中几乎无处不在，它们还依赖于通过组件运行时的二进制分析提取的数据。这些被称为伸缩分析的新方法结合了联合收割机收集极其详细的，关于系统的低级信息执行大量带有“大数据”的请求从详细观察的语料库中挖掘洞察力并创建系统操作模型的分析。 望远镜分析使用有针对性的、基于样本的日志记录和/或二进制分析来生成关于所观察系统的特定运行的大量高精度数据。 然后，它将这些观察结果结合到捕捉系统总体行为的模型中。将一般模型与每次运行的详细观察结果进行比较，可以了解该运行如何符合或偏离系统的一般操作。 该项目还在开发工具和查询语言，以便了解此类比较的结果，无论是在总体上还是与特定运行有关，用于性能分析，调试数据质量故障，了解离群行为，以及执行“假设”分析。

This project is developing scalable mechanisms to debug, monitor, andassess the quality of the complex distributed systems that representthe backbone of modern software infrastructure. These methods arenecessarily highly-automated; they reason about the operation ofdistributed systems while treating the components of such systems asblack boxes. This means that the methods do not require source code,programmer annotation, or developer input to troubleshoot adistributed system. Instead, they rely on detailed informationgleaned from pre-existing log messages that are nearly ubiquitous inevery large-scale distributed system and data extracted via binaryanalysis of components as they run.These new methods, termed telescopic analysis, combine the ability tocollect extremely detailed, low-level information about systemsexecuting large numbers of requests with "big data" analysis thatmines insights and create models of system operation from the corpusof detailed observations. Telescopic analysis uses targeted,sample-based logging and/or binary analysis to generate substantialquantities of high-precision data about specific runs of the systemunder observation. It then combines these observations into modelsthat capture the aggregate behavior of the system. Comparing thegeneral model with the detailed observations of each run allowsunderstanding of how that run conforms to or deviates from the commonoperation of the system. The project is also developing tools andquery languages that allow understanding of the results of suchcomparisons, both in aggregate and as pertains to specific runs, forperformance analysis, debugging data quality failures, understandingoutlier behavior, and performing "what-if" analysis.