Performance Tuning of MapReduce Jobs Using Surrogate-based Modeling
Performance Tuning of MapReduce Jobs Using Surrogate-based Modeling
复制标题
DOI：
10.1016/j.procs.2015.05.193
复制
发表时间：
2015-09
影响因子：
2.4
通讯作者：
Travis Johnston;Mohammad Alsulmi;Pietro Cicotti;M. Taufer
中科院分区：
文献类型：
--
作者：
Travis Johnston;Mohammad Alsulmi;Pietro Cicotti;M. Taufer
工作流性能建模对于寻找最佳配置参数和优化执行时间至关重要。我们将基于代理的建模方法应用于 MapReduce 作业的性能调整。我们构建一个由多元多项式定义的代理模型，其中包含每个要调整的参数的变量。出于说明目的，我们仅关注两个参数：并行映射器的数量和并行减速器的数量。我们证明可以通过对一小组参数空间进行采样来建立准确的性能模型。我们比较使用不同采样方法以及不同建模方法时构建模型的准确性和成本。我们得出的结论是，我们描述的基于代理的方法比其他众所周知的调整方法在采样时间方面更便宜，而且更准确。
Modeling workflow performance is crucial for finding optimal configuration parameters and optimizing execution times. We apply the method of surrogate-based modeling to performance tuning of MapReduce jobs. We build a surrogate model defined by a multivariate polynomial containing a variable for each parameter to be tuned. For illustrative purposes, we focus on just two parameters: the number of parallel mappers and the number of parallel reducers. We demonstrate that an accurate performance model can be built sampling a small set of the parameter space. We compare the accuracy and cost of building the model when using different sampling methods as well as when using different modeling approaches. We conclude that the surrogate-based approach we describe is both less expensive in terms of sampling time and more accurate than other well-known tuning methods.