Towards effective assessment of steady state performance in Java software: are we there yet?
Towards effective assessment of steady state performance in Java software: are we there yet?
复制标题
有效评估 Java 软件稳态性能：我们到了吗？
DOI：
10.1007/s10664-022-10247-x
复制
发表时间：
2022
影响因子：
4.1
通讯作者：
Michele Tucci
中科院分区：
文献类型：
--
作者：
L. Traini;V. Cortellessa;Daniele Di Pompeo;Michele Tucci
微基准测试是Java软件中广泛使用的性能测试形式。微基准测试重复执行一小块代码，同时收集与其性能相关的度量。由于Java虚拟机优化，微基准测试通常在执行的第一阶段（也称为预热）会受到严重的性能波动的影响。由于这个原因，软件开发人员通常放弃这个阶段的测量，并在基准达到稳定的性能状态时集中进行分析。开发人员根据他们的专业知识估计预热阶段的结束，并相应地配置他们的基准测试。不幸的是，这种方法基于两个强有力的假设：（i）基准测试总是达到稳定的性能状态，（ii）开发人员准确地估计预热。在本文中，我们表明，Java微基准测试并不总是达到稳定状态，往往开发人员无法准确地估计结束的热身阶段。我们发现，相当一部分研究的基准没有达到稳定状态，软件开发人员提供的预热估计往往是不准确的（有很大的误差）。这对成果质量和时间-努力都有重大影响。此外，我们发现，动态重新配置显着提高预热估计精度，但它仍然会导致次优预热估计和相关的副作用。我们设想这篇论文作为一个起点，支持引入更复杂的自动化技术，可以确保结果的质量及时。
Microbenchmarking is a widely used form of performance testing in Java software. A microbenchmark repeatedly executes a small chunk of code while collecting measurements related to its performance. Due to Java Virtual Machine optimizations, microbenchmarks are usually subject to severe performance fluctuations in the first phase of their execution (also known as warmup). For this reason, software developers typically discard measurements of this phase and focus their analysis when benchmarks reach a steady state of performance. Developers estimate the end of the warmup phase based on their expertise, and configure their benchmarks accordingly. Unfortunately, this approach is based on two strong assumptions: (i) benchmarks always reach a steady state of performance and (ii) developers accurately estimate warmup. In this paper, we show that Java microbenchmarks do not always reach a steady state, and often developers fail to accurately estimate the end of the warmup phase. We found that a considerable portion of studied benchmarks do not hit the steady state, and warmup estimates provided by software developers are often inaccurate (with a large error). This has significant implications both in terms of results quality and time-effort. Furthermore, we found that dynamic reconfiguration significantly improves warmup estimation accuracy, but still it induces suboptimal warmup estimates and relevant side-effects. We envision this paper as a starting point for supporting the introduction of more sophisticated automated techniques that can ensure results quality in a timely fashion.
影响因子：
--
作者：
Edd Barrett;Carl Friedrich Bolz-Tereick;Rebecca Killick;S. Mount;L. Tratt
通讯作者：
Edd Barrett;Carl Friedrich Bolz-Tereick;Rebecca Killick;S. Mount;L. Tratt
DOI：
--
发表时间：
2012
期刊：
--
影响因子：
--
作者：
Kalibera, T
通讯作者：
Kalibera, T
影响因子：
1.1
作者：
Araújo-Filho, Irami;Rêgo, Amália Cínthia Meneses;Medeiros, Aldo Cunha
通讯作者：
Medeiros, Aldo Cunha