Performance and Monetary Cost of Large-Scale Distributed Graph Processing on Amazon Cloud
Performance and Monetary Cost of Large-Scale Distributed Graph Processing on Amazon Cloud
复制标题
亚马逊云上大规模分布式图形处理的性能和货币成本
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
R. Goh
中科院分区：
文献类型：
--
作者：
Zengxiang Li;T. Hung;Sifei Lu;R. Goh
图形分析已成为揭示复杂系统中关系洞察的关键。随着图形规模的扩大，基于商用计算机和/或云实例开发了几个图形并行框架，包括Pregel、GraphLab和PowerGraph。根据最近的研究和经验性能评估，PowerGraph上的系统优化使其在处理具有偏度分布的自然图时的性能显著优于其他方法。然而，其性能特征、资源使用模式和货币成本却一直没有得到深入的探讨。本文使用三种不同的算法在高达768个CPU核的Amazon EC2实例上对PowerGraph进行了评估。我们发现，图形处理性能并不总是随着云资源的增加而提高。由于同步开销，资源没有得到充分利用。图形处理任务可能会选择不同的执行策略，指定数量和类型的云实例，以获得高性价比，在金钱成本和执行性能之间进行权衡。
Graph analytics has become essential to uncover relationship insights in complex systems. As graphs grow in scale, several graph-parallel frameworks including Pregel, GraphLab, and PowerGraph are developed based on commodity computers and/or Cloud instances. According to recent research and empirical performance evaluation, system optimization on PowerGraph allow it to outperform others significantly for processing natural graphs with skewed degree distribution. However, the performance characters, resource usage pattern and monetary cost have never been explored in-depth. In this paper, PowerGraph are evaluated with three different algorithms on Amazon EC2 instances with upto 768 CPU cores. We find that thegraph processing performance does not always increase with the increasing Cloud resources. Due to synchronization overheads, resources are not fully utilized. Graph processing tasks may prefer different execution strategies with specified number and type of Cloud instances to achieve high cost-efficiency, playing the trade-off between monetary cost and execution performance.