Efficient Experiment Selection in Automated Software Performance Evaluations
Efficient Experiment Selection in Automated Software Performance Evaluations
复制标题
自动化软件性能评估中的高效实验选择
DOI：
10.1007/978-3-642-24749-1_24
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
J. Happe
中科院分区：
文献类型：
--
作者：
D. Westermann;Rouven Krebs;J. Happe
当今企业应用程序的性能受到跨不同层的各种参数的影响。因此，评估这些系统的性能是一个耗费时间和资源的过程。可能的参数组合和配置的数量需要许多实验才能得出有意义的结论。尽管有许多自动化性能测试的工具可用，但是控制实验和分析结果仍然需要大量的手工工作。在本文中，我们应用统计模型推理技术，即Kriging和MARS，以自适应选择实验。我们的方法根据从当前可用数据推断出的模型所观察到的精度自动选择并进行实验。我们使用工业ERP场景验证了该方法。结果表明，仅使用配置空间中18%的测点，我们就可以自动推断出平均相对误差为1.6%的预测模型。
The performance of today's enterprise applications is influenced by a variety of parameters across different layers. Thus, evaluating the performance of such systems is a time and resource consuming process. The amount of possible parameter combinations and configurations requires many experiments in order to derive meaningful conclusions. Although many tools for automated performance testing are available, controlling experiments and analyzing results still requires large manual effort. In this paper, we apply statistical model inference techniques, namely Kriging and MARS, in order to adaptively select experiments. Our approach automatically selects and conducts experiments based on the accuracy observed for the models inferred from the currently available data. We validated the approach using an industrial ERP scenario. The results demonstrate that we can automatically infer a prediction model with a mean relative error of 1.6% using only 18% of the measurement points in the configuration space.