Dapper, a Large-Scale Distributed Systems Tracing Infrastructure
Dapper, a Large-Scale Distributed Systems Tracing Infrastructure
复制标题
DOI：
--
复制
发表时间：
2010
期刊：
影响因子：
--
通讯作者：
B. Sigelman;L. Barroso;M. Burrows;Patrick Stephenson;Manoj Plakal;Donald Beaver;Saul Jaspan;C. Shanbh
中科院分区：
文献类型：
--
作者：
B. Sigelman;L. Barroso;M. Burrows;Patrick Stephenson;Manoj Plakal;Donald Beaver;Saul Jaspan;C. Shanbh
现代互联网服务通常被实现为复杂的、大规模的分布式系统。这些应用程序由不同团队开发的软件模块集合构建而成，可能使用不同的编程语言，并且可以跨越多个物理设施的数千台机器。在这样的环境中，帮助理解系统行为和推理性能问题的工具是非常宝贵的。在这里，我们介绍了Dapper，谷歌的生产分布式系统跟踪基础设施的设计，并描述了我们的设计目标，低开销，应用程序级的透明度，无处不在的部署在一个非常大规模的系统得到满足。Dapper与其他跟踪系统在概念上有相似之处，特别是Magpie [3]和X-Trace [12]，但某些设计选择是其在我们的环境中成功的关键，例如使用采样和将仪器限制在相当少的公共库中。本文的主要目标是报告我们两年多来构建、部署和使用该系统的经验，因为Dapper成功的最重要衡量标准是它对开发人员和运营团队的有用性。Dapper最初是一个独立的跟踪工具，但后来发展成为一个监控平台，可以创建许多不同的工具，其中一些是其设计者没有预料到的。我们描述了一些使用Dapper构建的分析工具，分享了有关其在Google中使用情况的统计数据，展示了一些示例用例，并讨论了迄今为止学到的经验教训。
Modern Internet services are often implemented as complex, large-scale distributed systems. These applications are constructed from collections of software modules that may be developed by different teams, perhaps in different programming languages, and could span many thousands of machines across multiple physical facilities. Tools that aid in understanding system behavior and reasoning about performance issues are invaluable in such an environment. Here we introduce the design of Dapper, Google’s production distributed systems tracing infrastructure, and describe how our design goals of low overhead, application-level transparency, and ubiquitous deployment on a very large scale system were met. Dapper shares conceptual similarities with other tracing systems, particularly Magpie [3] and X-Trace [12], but certain design choices were made that have been key to its success in our environment, such as the use of sampling and restricting the instrumentation to a rather small number of common libraries. The main goal of this paper is to report on our experience building, deploying and using the system for over two years, since Dapper’s foremost measure of success has been its usefulness to developer and operations teams. Dapper began as a self-contained tracing tool but evolved into a monitoring platform which has enabled the creation of many different tools, some of which were not anticipated by its designers. We describe a few of the analysis tools that have been built using Dapper, share statistics about its usage within Google, present some example use cases, and discuss lessons learned so far.