A Pareto Framework for Data Analytics on Heterogeneous Systems: Implications for Green Energy Usage and Performance
A Pareto Framework for Data Analytics on Heterogeneous Systems: Implications for Green Energy Usage and Performance
复制标题
DOI：
10.1109/icpp.2017.62
复制
发表时间：
2017-08
期刊：
影响因子：
--
通讯作者：
Aniket Chakrabarti;S. Parthasarathy;Christopher Stewart
中科院分区：
文献类型：
--
作者：
Aniket Chakrabarti;S. Parthasarathy;Christopher Stewart
用于数据分析的分布式算法将其输入数据划分到多台机器上，以便并行执行。在规模上，有些机器的性能可能会比其他机器差，因为它们速度较慢，功率受限或依赖于不受欢迎的脏能源。在异构机器上平衡分析工作负载具有挑战性，因为算法对数据分区中的统计偏差很敏感。倾斜的分区会降低整个工作负载的速度或降低结果的质量。根据每台机器的性能来调整分区的大小可能会引入或进一步加剧偏斜。在本文中，我们提出了一个计划，控制每个分区的统计分布和大小分区根据计算环境的异构性。我们将异构性建模为多目标优化，目标是执行时间和脏能耗的函数。我们使用分层来控制偏斜。实验表明，我们的计算异质性感知（Het-Aware）分区策略加快了运行时间高达51%的分层分区方案基线。我们还有一个异构性和能源感知（Het-Energy-Aware）分区方案，它比Het-Aware解决方案慢，但可以将脏能源足迹降低高达26%。对于一些分析任务，使用这种划分策略也有显着的质量优势。
Distributed algorithms for data analytics partition their input data across many machines for parallel execution. At scale, it is likely that some machines will perform worse than others because they are slower, power constrained or dependent on undesirable, dirty energy sources. It is challenging to balance analytics workloads across heterogeneous machines because the algorithms are sensitive to statistical skew in data partitions. A skewed partition can slow down the whole workload or degrade the quality of results. Sizing partitions in proportion to each machine's performance may introduce or further exacerbate skew. In this paper, we propose a scheme that controls the statistical distribution of each partition and sizes partitions according to the heterogeneity of the computing environment. We model heterogeneity as a multi-objective optimization, with the objectives being functions for execution time and dirty energy consumption. We use stratification to control skew. Experiments show that our computational heterogeneity-aware (Het-Aware) partitioning strategy speeds up running time by up to 51% over the stratified partitioning scheme baseline. We also have a heterogeneity and energy aware (Het-Energy-Aware) partitioning scheme which is slower than the Het-Aware solution but can lower the dirty energy footprint by up to 26%. For some analytic tasks, there is also a significant qualitative benefit when using such partitioning strategies.