XPS: CLCCA: On the Hunt for Correctness and Performance Bugs in Large-scale Programs

在过去的几年中，计算应用程序的规模急剧增加。随着计算基因组学、数据挖掘和机器学习等领域的应用程序在越来越复杂的问题上得到释放，这些应用程序的输入规模也在迅速增长。由于对并行性的追求导致了服务器核心数量的增加，以及数据中心服务器和机架数量的增加，这些应用程序必须运行的系统的规模也急剧增加。开发大规模应用程序的一个关键问题是检测和调试可伸缩性问题，这些问题是只有当程序扩展时才会出现的程序行为问题。伸缩性问题表现为正确性错误或性能瓶颈。不幸的是，检测大规模出现的bug是很困难的。手动浏览日志或对单个应用程序进程进行性能分析是不实际的。此外，开发人员可能无法访问大规模运行应用程序所需的输入和系统。本研究项目旨在开发自动化技术，通过程序行为建模、小规模运行训练和大规模运行推断来检测和诊断大规模程序的正确性和性能错误。为了实现我们的目标，我们建立了包含规模的统计模型。通过将程序规模与程序行为联系起来，我们可以预测程序在大范围内的行为，而无需在该范围内看到正确的行为，并使用这些预测来检测和诊断错误。该项目围绕三个重点进行构建，每个重点都使用计算基因组学应用程序作为上下文。首先，我们建立了包含规模的程序行为的统计模型。在第二部分中，我们构建了用于检测错误的统计技术，然后深入挖掘以识别软件中潜在的根本原因。在第三部分中，我们构建了一个测试工具，它将允许我们以加速的方式发现此类扩展问题。总的来说，该项目以创新的方式结合了静态分析、动态仪器、建模和基于机器学习的数据分析的应用。该项目将使用计算基因组学应用程序（如Blast、Bowtie、Trinity/Butterfly和Margin）来评估该方法。

The scale of computing applications has been dramatically increasing over the past several years. As applications in domains such as computational genomics, data mining, and machine learning are let loose on ever-more-complex problems, the scale of the inputs to these applications has shot up. And as the pursuit of parallelism has led to increasing core counts for servers, and increasing numbers of servers and racks for data centers, the scale of the systems that these applications must run on has also dramatically risen. A critical problem in developing large scale applications is detecting and debugging scaling issues, which are problems with program behavior that emerge only as a program scales up. Scaling issues show up as correctness bugs or performance bottlenecks. Unfortunately, detecting bugs that arise at large scales is difficult. Manually poring through logs or performance profiling individual application processes is not practical. Moreover, the developer may not have access to the inputs and systems necessary to run the application at large scales. This research project aims to develop automated techniques to detect and diagnose correctness and performance bugs for large-scale programs using program behavior modeling, training at small scale runs, and extrapolating to large-scale runs.To achieve our objectives, we build statistical models that incorporate scale. By relating program scale to program behavior, we can predict how a program behaves at large scales, without ever seeing correct behavior at that scale, and use those predictions to detect and diagnose bugs. The project is structured around three thrusts, each using the computational genomics applications for context. In the first, we build statistical models of program behavior that incorporate scale. In the second, we build statistical techniques for detecting when there is an error and then drilling down to identify potential root causes in the software. In the third, we build a testing tool which will allow us to uncover such scaling issues in an accelerated manner. In aggregate, the project combines in innovative ways applications of static analysis, dynamic instrumentation, modeling, and machine learning-based data analysis. The project will use computational genomics applications, such as Blast, Bowtie, Trinity/Butterfly, and Margin, to evaluate the approach.