XPS: CLCCA: On the Hunt for Correctness and Performance Bugs in Large-scale Programs

计算应用的规模在过去几年中急剧增加。随着计算基因组学、数据挖掘和机器学习等领域的应用程序在越来越复杂的问题上发挥作用，这些应用程序的输入规模急剧上升。随着对并行性的追求导致服务器核心数量的增加，以及数据中心服务器和机架数量的增加，这些应用程序必须运行的系统规模也急剧增加。开发大规模应用程序的一个关键问题是检测和调试缩放问题，这是程序行为的问题，只有当程序按比例增加时才会出现。扩展问题表现为正确性错误或性能瓶颈。不幸的是，检测大规模出现的错误是困难的。手动查看日志或对单个应用程序进程进行性能分析是不切实际的。此外，开发人员可能无法访问大规模运行应用程序所需的输入和系统。本研究项目旨在开发自动化技术，通过程序行为建模，在小规模运行时进行训练，并外推到大规模运行，来检测和诊断大规模程序的正确性和性能错误。为了实现我们的目标，我们构建了包含规模的统计模型。通过将程序规模与程序行为联系起来，我们可以预测程序在大规模下的行为，而不会看到该规模下的正确行为，并使用这些预测来检测和诊断错误。该项目围绕三个重点展开，每个重点都使用计算基因组学应用程序作为背景。在第一，我们建立的程序行为，纳入规模的统计模型。在第二部分中，我们构建了统计技术，用于检测何时出现错误，然后深入挖掘以确定软件中的潜在根本原因。第三，我们构建了一个测试工具，这将使我们能够以更快的方式发现这种扩展问题。 总的来说，该项目以创新的方式结合了静态分析，动态仪器，建模和基于机器学习的数据分析的应用。 该项目将使用计算基因组学应用程序，如Blast，Bowtie，Trinity/Butterfly和Margin来评估该方法。

The scale of computing applications has been dramatically increasing over the past several years. As applications in domains such as computational genomics, data mining, and machine learning are let loose on ever-more-complex problems, the scale of the inputs to these applications has shot up. And as the pursuit of parallelism has led to increasing core counts for servers, and increasing numbers of servers and racks for data centers, the scale of the systems that these applications must run on has also dramatically risen. A critical problem in developing large scale applications is detecting and debugging scaling issues, which are problems with program behavior that emerge only as a program scales up. Scaling issues show up as correctness bugs or performance bottlenecks. Unfortunately, detecting bugs that arise at large scales is difficult. Manually poring through logs or performance profiling individual application processes is not practical. Moreover, the developer may not have access to the inputs and systems necessary to run the application at large scales. This research project aims to develop automated techniques to detect and diagnose correctness and performance bugs for large-scale programs using program behavior modeling, training at small scale runs, and extrapolating to large-scale runs.To achieve our objectives, we build statistical models that incorporate scale. By relating program scale to program behavior, we can predict how a program behaves at large scales, without ever seeing correct behavior at that scale, and use those predictions to detect and diagnose bugs. The project is structured around three thrusts, each using the computational genomics applications for context. In the first, we build statistical models of program behavior that incorporate scale. In the second, we build statistical techniques for detecting when there is an error and then drilling down to identify potential root causes in the software. In the third, we build a testing tool which will allow us to uncover such scaling issues in an accelerated manner. In aggregate, the project combines in innovative ways applications of static analysis, dynamic instrumentation, modeling, and machine learning-based data analysis. The project will use computational genomics applications, such as Blast, Bowtie, Trinity/Butterfly, and Margin, to evaluate the approach.