Pervolution: Performance Evolution of Highly-Configurable Software Systems

点击翻译按钮获取中文摘要

A software system is not a static entity. Instead, it changes all the time through bug fixes, refactorings, and the addition of new features. All these code changes can have a profound effect on performance, possibly causing performance bugs or inefficient computations. Even worse, performance changes often manifest only for certain software configurations. That is, depending on how a software system is configured, we may observe a slowdown in performance or not. Detecting such changes across the entire project history and framing them as evolutionary patterns has been the goal of Pervolution's first phase.The developed methods and tools for static code analysis and evolutionary performance models build the backbone for the second project phaseThe goal of Pervolution's second phase is to reliably determine root causes of evolutionary performance changes of configurable softwaresystems under varying workloads. Addressing this goal generalizes the results of the first phase to a practical setting where over theproject's lifetime not only the software changes, but also the input and the use cases of the software system. Hence, we extend the realm of performance modelling to a three-dimensional entity: space dimension (configuration), time dimension (versions), and workload dimension (system input). In this space, we aim at finding evolutionary patterns for performance changes and their causes, such as which kinds of workload-dependent configuration options may cause a performance bug that is introduced with certain kinds of commits.Since this combined space is too large to effectively sample and learn performance models, our approach is threefold: First, we extend our static code analysis from the first phase with black-box information to scale the analysis to larger systems and project histories. This way, we can use data-flow analysis to point to options and combinations thereof whose execution and data-flow have changed across versions, possibly affecting the performance behavior. Second, we extend or sampling and learning techniques to account for workload variation, for example, by applying transfer learning. Third, we analyze and model a system’s internal performance-relevent factors such that we are able to learn causal chains from configuration options over internal metrics (e.g., cache misses) toperformance data. Here, we apply and adapt state-of-the-art techniques from causal learning to the realm of performance modelling ofconfigurable software systems. Combing all three lanes will allows us at the end of Pervolution to answer questions, such as will an older version of a feature perform faster of a certain workload or was the introduction of the new option the root-cause of an observed performance bug.