CAREER: Understanding the Performance of Distributed Systems through Causal Tracing

职业：通过因果追踪了解分布式系统的性能社会越来越依赖于大型的、分散的、有许多组件以复杂而微妙的方式相互作用的共享软件系统。这些系统包括金融和银行服务、网络和云资源、机票预订、大数据和科学计算等等。尽管它们的影响范围和成功是毋庸置疑的，但在这些系统中，很难回答有关故障原因的问题，发现组件之间的依赖问题，确定一个操作对系统其余部分的影响，或者向用户提供性能保证。通过开发和应用技术来实现对大规模分布式系统性能特征的深入和实时理解，该项目？S的目标是开发技术，使这些系统的用户和提供者能够更好地表达他们在性能方面的需求和保证，并更好地计划和减轻故障的影响。这个项目的主要观点是，由于分布式系统中有许多组件，在一个组件中发起的操作的上下文会随着操作涉及到其他组件而丢失。这使得系统深处的组件很难辨别它正在与哪个客户机一起工作，也很难应用一致的策略或考虑跨组件边界的操作成本。本研究将创建跟踪平面的抽象，该平面在整个系统执行过程中保留此上下文，允许调试和诊断性能问题，并实时提供性能保证。这个跟踪平面将是一个无处不在的基础设施，用于从分布式系统的执行中收集因果信息，并促进分析和诊断任务的有效部署。此外，通过以一致的方式跨所有组件聚合有关系统中任务的信息，跟踪平面能够实现可以在本地、实时和具有全局知识的情况下执行的资源管理策略——这在目前是不可能的。今天，我们更擅长于构建大规模分布式系统，而不是精确地理解它们是如何工作的，以及它们是如何失败的，这将提供一个核心的教育方面，因为跟踪平面是一个强大的教学工具，用于理解分布式系统的结构和执行。这项工作将吸引本科生和研究生，以及操作这种大规模分布式系统的行业合作伙伴。从增加对这些系统的可见性开始，这个项目的最终目标是提供工具和方法，以允许构建、操作和管理高效、可靠和可预测的大规模共享分布式系统。随着社会越来越依赖于这类系统，这项研究具有巨大而持久的潜在影响。

CAREER: Understanding the Performance of Distributed Systems through Causal Tracing Society increasingly depends on shared software systems that are large, decentralized, with many components that interact in complex and subtle ways. These systems include financial and banking services, Web and cloud resources, airline reservations, and big data and scientific computing, to name a few. Despite their unquestionable reach and success, in these systems it is very hard to answer questions about the causes of failures, to uncover dependency issues among their components, to determine the impact of one operation on the rest of the system, or to provide guarantees about their performance to users. By developing and applying techniques to enable deep and real-time understanding of the performance characteristics of large-scale distributed systems, this project?s goal is to develop techniques that will enable users and providers of these systems to better express their needs and their guarantees in terms of performance, and better plan for and mitigate the effects of failures. The main insight in this project is that because of the many components in distributed systems, the context of an operation initiated in one component gets lost as the operation involves other components. This makes it hard for a component deep in the system to discern with which client it is working, making it also hard to apply consistent policies or account for the cost of operations across component boundaries. This research will create the abstraction of a Tracing Plane that preserves this context throughout the entire execution of the system, allowing for debugging and diagnosis of performance problems, and for real-time provisioning of performance guarantees. This Tracing Plane will be a pervasive infrastructure to collect causal information from the execution of a distributed system and facilitate the efficient deployment of analytics and diagnostic tasks. Further, by aggregating information about tasks in the system across all components in a coherent way, the Tracing Plane enables the implementation of resource management policies that can act locally, in real-time, and with global knowledge - which is presently not possible. We are better today at building large-scale distributed systems than we are at understanding precisely how they work, and how they fail and this will provide a core educational aspect, as the Tracing Plane is a strong pedagogical tool for the understanding of distributed systems structure and execution. This work will engage undergraduate and graduate students, as well as industry partners that operate such large-scale distributed systems. By starting from increased visibility into these systems, the ultimate goal of this project is to provide tools and methods to allow building, operation, and management of large-scale, shared distributed systems that are efficient, reliable, and predictable. As society increasingly depends on systems of this kind, this research has a large and long lasting potential impact.