New Scheduling Algorithm in Hadoop Based on Resource Aware

New Scheduling Algorithm in Hadoop Based on Resource Aware
复制标题

DOI：
10.1007/978-3-642-54927-4_97
复制
发表时间：
2014
期刊：
--
影响因子：
--
通讯作者：
Peng Xu;H. Wang;Ming Tian
Peng Xu;H. Wang;Ming Tian
中科院分区：
其他
文献类型：
--
作者：
Peng Xu;H. Wang;Ming Tian

文献摘要

被引文献

相似文献

Hadoop MapReduce is a popular distributed computing model that has been deployed on large clusters like those owned by Yahoo and Facebook. In a practical data center of that scale, it is a common scenario that I/O bound jobs and CPU bound jobs, that demand complementary resources, run simultaneously on the same cluster. In current Hadoop scheduling algorithm, parallelization of these two kinds of job has not been concerned. In this paper, we propose a new resource aware scheduling algorithm. With this algorithm we can classify the type of work and node workload reasonably, and the algorithm mainly solve the problems that affect the processing rate which arise during resource contention. Finally, the experiments and performance analysis of the algorithm proved that the algorithm could significantly improve the system’s throughput.