Java performance evaluation through rigorous replay compilation
Java performance evaluation through rigorous replay compilation
复制标题
通过严格的重放编译评估Java性能
DOI：
--
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
D. Buytaert
中科院分区：
文献类型：
--
作者：
A. Georges;L. Eeckhout;D. Buytaert
诸如Java Virtual Machine之类的托管运行时环境对基准不是平底的。 Java性能受到应用程序及其输入以及虚拟机（JIT优化器，垃圾收集器，线程调度程序等）的影响。此外，由于基于计时器的采样，用于JIT优化，线程调度和各种系统效应引起的非确定性使Java性能基准测试过程更加复杂。重播汇编是最近引入的Java性能分析方法，旨在控制非确定性以提高实验性重复性。重播汇编的关键思想是通过在重播时诱导预录的汇编计划来控制实验期间的汇编负载。重播汇编还可以使应用程序与虚拟机的性能效果分开。本文认为，与当前使用单个汇编计划的实践相反，多个汇编计划为重播汇编方法增加了统计严格。通过这样做，重播汇编可以更好地说明跨汇编计划中汇编负载中观察到的可变性。此外，我们建议对统计数据分析的匹配对比较。匹配的比较比较考虑了兴趣创新作为一对创新之前和之后的每个汇编计划的绩效测量，从而限制了与统计分析相比，假设未配对的测量值，则可以限制准确绩效分析所需的汇编计划的数量。
A managed runtime environment, such as the Java virtual machine, is non-trivial to benchmark. Java performance is affected in various complex ways by the application and its input, as well as by the virtual machine (JIT optimizer, garbage collector, thread scheduler, etc.). In addition, non-determinism due to timer-based sampling for JIT optimization, thread scheduling, and various system effects further complicate the Java performance benchmarking process. Replay compilation is a recently introduced Java performance analysis methodology that aims at controlling non-determinism to improve experimental repeatability. The key idea of replay compilation is to control the compilation load during experimentation by inducing a pre-recorded compilation plan at replay time. Replay compilation also enables teasing apart performance effects of the application versus the virtual machine. This paper argues that in contrast to current practice which uses a single compilation plan at replay time, multiple compilation plans add statistical rigor to the replay compilation methodology. By doing so, replay compilation better accounts for the variability observed in compilation load across compilation plans. In addition, we propose matched-pair comparison for statistical data analysis. Matched-pair comparison considers the performance measurements per compilation plan before and after an innovation of interest as a pair, which enables limiting the number of compilation plans needed for accurate performance analysis compared to statistical analysis assuming unpaired measurements.