XPS: CLCCA: On the Hunt for Correctness and Performance Bugs in Large-scale Programs

在过去的几年里，计算应用程序的规模一直在急剧增加。随着计算基因组学、数据挖掘和机器学习等领域的应用程序被释放到越来越复杂的问题上，这些应用程序的投入规模急剧上升。随着对并行性的追求导致服务器的核心数量增加，以及数据中心的服务器和机架数量增加，这些应用程序必须在其上运行的系统的规模也大幅增加。开发大型应用程序的一个关键问题是检测和调试伸缩问题，这些问题是只有在程序扩展时才会出现的程序行为问题。扩展问题表现为正确性错误或性能瓶颈。不幸的是，检测大规模出现的错误是困难的。手动研究日志或分析单个应用程序进程的性能是不切实际的。此外，开发人员可能无法访问大规模运行应用程序所需的输入和系统。这项研究项目旨在开发自动化技术来检测和诊断大规模程序的正确性和性能错误，使用程序行为建模、小规模运行时的训练和大规模运行的外推。为了实现我们的目标，我们构建了纳入规模的统计模型。通过将程序规模与程序行为相关联，我们可以预测程序在大范围内的行为，而不需要在该规模上看到正确的行为，并使用这些预测来检测和诊断错误。该项目围绕三个方面展开，每一个方面都使用了计算基因组学应用程序作为背景。首先，我们建立了包含尺度的程序行为统计模型。在第二个步骤中，我们构建了统计技术，用于检测何时出现错误，然后向下钻取以确定软件中潜在的根本原因。第三，我们构建了一个测试工具，它将允许我们以更快的方式发现此类扩展问题。总体而言，该项目以创新的方式结合了静态分析、动态检测、建模和基于机器学习的数据分析的应用。该项目将使用计算基因组学应用程序，如BLAST、Bowtie、三一/蝴蝶和边际来评估该方法。

The scale of computing applications has been dramatically increasing over the past several years. As applications in domains such as computational genomics, data mining, and machine learning are let loose on ever-more-complex problems, the scale of the inputs to these applications has shot up. And as the pursuit of parallelism has led to increasing core counts for servers, and increasing numbers of servers and racks for data centers, the scale of the systems that these applications must run on has also dramatically risen. A critical problem in developing large scale applications is detecting and debugging scaling issues, which are problems with program behavior that emerge only as a program scales up. Scaling issues show up as correctness bugs or performance bottlenecks. Unfortunately, detecting bugs that arise at large scales is difficult. Manually poring through logs or performance profiling individual application processes is not practical. Moreover, the developer may not have access to the inputs and systems necessary to run the application at large scales. This research project aims to develop automated techniques to detect and diagnose correctness and performance bugs for large-scale programs using program behavior modeling, training at small scale runs, and extrapolating to large-scale runs.To achieve our objectives, we build statistical models that incorporate scale. By relating program scale to program behavior, we can predict how a program behaves at large scales, without ever seeing correct behavior at that scale, and use those predictions to detect and diagnose bugs. The project is structured around three thrusts, each using the computational genomics applications for context. In the first, we build statistical models of program behavior that incorporate scale. In the second, we build statistical techniques for detecting when there is an error and then drilling down to identify potential root causes in the software. In the third, we build a testing tool which will allow us to uncover such scaling issues in an accelerated manner. In aggregate, the project combines in innovative ways applications of static analysis, dynamic instrumentation, modeling, and machine learning-based data analysis. The project will use computational genomics applications, such as Blast, Bowtie, Trinity/Butterfly, and Margin, to evaluate the approach.