CSR: Small: A Just-in-Time, Cross-Layer Instrumentation Framework for Diagnosing Performance Problems in Distributed Applications

CSR：小型：用于诊断分布式应用程序中性能问题的即时跨层仪表框架

• 批准号: 1815323
• 负责人: Raja Sambasivan
• 金额: $ 46.02万
• 依托单位: Trustees of Boston University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2020-02-29
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815323&HistoricalAwards=false
• 关键词: CSR; Small; Just; Time; Cross

Distributed applications running in data centers are critical to society (e.g., for shopping, banking). Engineers must diagnose and fix problems observed in data centers quickly; however, doing so is extremely challenging. A significant hurdle is that engineers must spend significant time and effort exploring what instrumentation (e.g., log messages about specific application behaviors) is needed to provide visibility into a new problem. To assist in this front, this project will develop an instrumentation framework that, in response to a new problem, will automatically search the space of possible instrumentation choices and enable the instrumentation needed to provide insight into it.This project addresses fundamental challenges associated with creating an automatic instrumentation framework: (a) What algorithms and heuristics are suited for automatically and efficiently exploring the instrumentation search space? (b) What architectural support is needed within the framework to enable automatic exploration? (c) How can the search space be explored without significantly impacting application performance? The proposal will explore the utility of algorithms based on operator knowledge, statistics, and machine learning to explore the search space. It will build on end-to-end tracing, as this will enable the framework to work for problems that affect different sets of requests.This project will inform the architecture of next-generation instrumentation frameworks, which are needed to keep pace with the ever-increasing complexity of distributed applications. The critical issues identified in popular open-source distributed applications while evaluating the framework will improve their robustness. Researchers will be able to leverage the software artifacts released by this project to create novel distributed-application-management tools that leverage the framework's unique capabilities. They will be able to deploy the framework in research clouds to obtain valuable workload traces from them. The project will generate course modules on diagnosis practices for distributed applications.The artifacts produced by this project, including framework source code, workload traces, instrumented applications, and research results, will be freely disseminated online at: https://massopen.cloud and https://www.rajasambasivan.com. All software artifacts will be stored in Github as well. All artifacts will be available for a minimum of seven years from the start of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在数据中心运行的分布式应用对社会(例如，购物、银行)至关重要。工程师必须快速诊断和修复数据中心中观察到的问题；然而，这样做极具挑战性。一个重要的障碍是工程师必须花费大量的时间和精力来探索需要什么工具(例如，有关特定应用程序行为的日志消息)来提供对新问题的可见性。为了在这方面提供帮助，该项目将开发一个工具框架，该框架将响应一个新问题，自动搜索可能的工具选择空间，并使工具能够提供对它的洞察。本项目解决与创建自动工具框架相关的基本挑战：(A)哪些算法和启发式方法适合于自动有效地探索工具搜索空间？(B)框架内需要什么样的架构支持才能实现自动探索？(C)如何才能在不显著影响应用程序性能的情况下探索搜索空间？该提案将探索基于操作员知识、统计学和机器学习的算法的效用，以探索搜索空间。它将建立在端到端跟踪的基础上，因为这将使框架能够处理影响不同请求集的问题。该项目将向下一代工具框架的体系结构提供信息，这些框架是跟上分布式应用程序日益复杂的步伐所需的。在评估框架时，在流行的开源分布式应用程序中发现的关键问题将提高它们的健壮性。研究人员将能够利用该项目发布的软件构件来创建利用框架独特功能的新型分布式应用程序管理工具。他们将能够在研究云中部署该框架，以从他们那里获得有价值的工作负载跟踪。该项目将生成关于分布式应用程序诊断实践的课程模块。该项目产生的人工产物，包括框架源代码、工作负载跟踪、仪表化应用程序和研究结果，将在网上免费传播：https://massopen.cloud和https://www.rajasambasivan.com.所有软件产品也将存储在Github中。所有文物将在项目开始后至少七年内可用。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。