CAREER: Auto-generated experimentation for performance diagnosis of distributed systems

调试，在系统中发现和修复问题的过程，是计算机科学家最关键和最耗时的活动之一。本研究的重点是性能调试，这是最具挑战性的调试形式之一。性能调试很困难，因为减速通常不会在容易识别的位置破坏功能。诊断减速的位置及其原因需要收集和分析详细的性能度量。这对于只偶尔出现或只影响一小部分工作负载的减速来说尤其具有挑战性。再加上许多大大小小的公司都在构建由数百到数千个服务/组件组成的分布式系统，公司经常需要聘请专业的性能工程师团队来跟踪主要的性能问题也就不足为奇了。本研究的目标是开发用于自动诊断分布式系统中的性能问题的新工具和方法。本研究解决的不是识别有缺陷或行为不端的组件，而是识别系统设计和实现中根本低效的问题。这项研究将开创一种新的诊断方法，自动生成实验来验证或反驳性能假设。基于这些假设生成的实验将用于逐步缩小问题范围并确定减速的根本原因。由此产生的工具将为工程师提供深入了解调查的位置和内容，以便他们的工作将集中在解决问题而不是诊断问题上。这项研究的直接好处是开发了新的自动性能诊断方法和开源工具，以帮助普通软件开发人员和专业性能工程师找到系统中减速的来源。这节省了昂贵的工程时间，并可以帮助工程师建造成本更高、能效更高的系统。通过将代码分析和性能建模原则集成到自动化工具中，来自这项研究的想法更容易被更广泛的工程师所接受，否则这些工程师可能不具备这种专业知识。为了对调试方法和实践产生持久的影响，该项目还包括一个重要的教育组成部分，旨在通过以下方式改变本科课程中的调试教育：(i)开发一门新的调试课程，其中本研究的概念将被整合为课程模块；（ii）创建助教模块，培训助教如何进行调试教学。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Debugging, the process of finding and fixing problems in systems, is one of the most critical and time-consuming activities for computer scientists. This research focuses on performance debugging, one of the most challenging forms of debugging. Performance debugging is difficult because slowdowns typically do not break functionality in easily identifiable locations. Diagnosing where the slowdowns are, and their causes, requires gathering and analyzing detailed performance measurements. This is particularly challenging for slowdowns that only appear sporadically or only affect a fraction of the workload. Coupled with the fact that many large and small companies build distributed systems composed of hundreds to thousands of services/components, it is no surprise that companies often need to hire teams of specialized performance engineers to track down the main performance issues. The goal of this research is to develop new tools and methodologies for automatically diagnosing performance issues within distributed systems. Rather than identifying faulty or misbehaving components, this research tackles the harder problem of identifying fundamental inefficiencies within the design and implementation of a system. The research will pioneer a novel diagnosis approach that auto-generates experiments to validate or refute performance hypotheses. Experiments generated based on these hypotheses will be used to progressively narrow down the problem scope and identify the root cause(s) of slowdowns. The resulting tools will provide engineers insights into where and what to investigate so that their efforts will be focused on fixing problems rather than diagnosing them.The direct benefit of this research is in developing new automated performance diagnosis methodologies and open-sourced tools for assisting both general software developers and specialized performance engineers in finding sources of slowdowns in their systems. This saves costly engineering time and could help engineers build more cost- and energy-efficient systems. By integrating code analysis and performance modeling principles into the automated tool, the ideas from this research are more easily accessible to a broader base of engineers that might not otherwise have this specialized knowledge. To have a lasting effect on debugging methodologies and practices, this project also includes a significant education component that aims to transform debugging education in undergraduate curricula through (i) developing a new debugging course, where concepts from this research will be integrated as a course module; and (ii) creating a teaching assistant module for training teaching assistants on how to teach debugging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.