The Optimization of Memory Access Congestion for MapReduce Applications on Manycore Systems
The Optimization of Memory Access Congestion for MapReduce Applications on Manycore Systems
复制标题
众核系统上MapReduce应用的内存访问拥塞优化
DOI：
10.1093/comjnl/bxu136
复制
发表时间：
2016-03
期刊：
影响因子：
--
通讯作者：
Zhengdong Zhu
中科院分区：
文献类型：
--
作者：
Liang Li;Endog Wang;xiaoshe Dong;Zhengdong Zhu
The prevalence of heterogeneous manycore processors has been shown as a promising alternative to web-based data parallel MapReduce applications. The differences of manycore from multicore raise new challenges to designing and implementing efficient and scalable MapReduce applications, such as memory access congestion. This paper argues that it is more efficient using multiple small groups of cores than using a single group with all cores to address these challenges. We propose a group-based MapReduce implementation, called Grouped-MapReduce (GMR). It uses grouping to align tasks so that the tasks in each group are congestion-free. Further, it uses multiplexing to control the implementation of groups so that multiple groups can efficiently share the limited memory bandwidth without causing serious memory access congestion. We have implemented a prototype of GMR based on Phoenix++, an already highly optimized MapReduce runtime. Experiments on six benchmarks show that GMR implements and scales well on manycore systems and obtains an impressive improvement over Phoenix++ from 1.04x to 1.77x without artificially tuning the existing application code.
