Pervolution: Performance Evolution of Highly-Configurable Software Systems

软件系统不是一个静态的实体。相反，它通过bug修复、重构和添加新功能而不断变化。所有这些代码更改都可能对性能产生深远的影响，可能导致性能错误或低效计算。更糟糕的是，性能变化通常只表现在某些软件配置上。也就是说，根据软件系统的配置方式，我们可能会观察到性能是否下降。在整个项目历史中检测这些变化并将其作为进化模式一直是Pervolution第一阶段的目标。开发的静态代码分析和进化性能模型的方法和工具为第二阶段的项目构建了支柱Pervolution第二阶段的目标是可靠地确定可配置软件系统在不同工作负载下进化性能变化的根本原因。为了实现这个目标，第一阶段的结果被推广到一个实际的环境中，在项目的生命周期中，不仅软件会发生变化，而且软件系统的输入和用例也会发生变化。因此，我们将性能建模的领域扩展到三维实体：空间维度（配置），时间维度（版本）和工作负载维度（系统输入）。在这个空间中，我们的目标是找到性能变化的演化模式及其原因，例如哪些类型的工作负载相关的配置选项可能会导致某些类型的提交所引入的性能错误。由于这个组合空间太大，无法有效地采样和学习性能模型，我们的方法有三个方面：首先，我们从第一阶段扩展我们的静态代码分析，使用黑盒信息将分析扩展到更大的系统和项目历史。这样，我们就可以使用数据流分析来指出那些在不同版本中执行和数据流发生了变化的选项及其组合，从而可能影响性能行为。其次，我们扩展或采样和学习技术，以考虑工作量的变化，例如，通过应用迁移学习。第三，我们分析和建模系统的内部性能相关因素，以便我们能够从内部度量的配置选项中学习因果链（例如，高速缓存未命中）到性能数据。在这里，我们应用和适应国家的最先进的技术，从因果学习领域的性能建模ofconfigurable软件系统。将所有三个通道组合起来将使我们能够在Pervolution结束时回答问题，例如旧版本的功能是否会在特定工作负载下执行得更快，或者新选项的引入是否是观察到的性能错误的根本原因。

A software system is not a static entity. Instead, it changes all the time through bug fixes, refactorings, and the addition of new features. All these code changes can have a profound effect on performance, possibly causing performance bugs or inefficient computations. Even worse, performance changes often manifest only for certain software configurations. That is, depending on how a software system is configured, we may observe a slowdown in performance or not. Detecting such changes across the entire project history and framing them as evolutionary patterns has been the goal of Pervolution's first phase.The developed methods and tools for static code analysis and evolutionary performance models build the backbone for the second project phaseThe goal of Pervolution's second phase is to reliably determine root causes of evolutionary performance changes of configurable softwaresystems under varying workloads. Addressing this goal generalizes the results of the first phase to a practical setting where over theproject's lifetime not only the software changes, but also the input and the use cases of the software system. Hence, we extend the realm of performance modelling to a three-dimensional entity: space dimension (configuration), time dimension (versions), and workload dimension (system input). In this space, we aim at finding evolutionary patterns for performance changes and their causes, such as which kinds of workload-dependent configuration options may cause a performance bug that is introduced with certain kinds of commits.Since this combined space is too large to effectively sample and learn performance models, our approach is threefold: First, we extend our static code analysis from the first phase with black-box information to scale the analysis to larger systems and project histories. This way, we can use data-flow analysis to point to options and combinations thereof whose execution and data-flow have changed across versions, possibly affecting the performance behavior. Second, we extend or sampling and learning techniques to account for workload variation, for example, by applying transfer learning. Third, we analyze and model a system’s internal performance-relevent factors such that we are able to learn causal chains from configuration options over internal metrics (e.g., cache misses) toperformance data. Here, we apply and adapt state-of-the-art techniques from causal learning to the realm of performance modelling ofconfigurable software systems. Combing all three lanes will allows us at the end of Pervolution to answer questions, such as will an older version of a feature perform faster of a certain workload or was the introduction of the new option the root-cause of an observed performance bug.