Energy Utilization Task Scheduling for MapReduce in Heterogeneous Clusters
Energy Utilization Task Scheduling for MapReduce in Heterogeneous Clusters
复制标题
DOI：
10.1109/tsc.2020.2966697
复制
发表时间：
2022-03
影响因子：
8.1
通讯作者：
Jia Wang;Xiaoping Li;Rubén Ruiz;Jie Yang;Dianhui Chu
中科院分区：
文献类型：
--
作者：
Jia Wang;Xiaoping Li;Rubén Ruiz;Jie Yang;Dianhui Chu
如今，能源成本是云计算中最重要的因素。因此，实现能量感知的任务调度方法至关重要。提出了一种考虑截止时间、数据局部性和资源利用率的任务调度框架，以节省异构集群中的能源成本。该框架由任务列表构建、任务调度和槽列表更新组成。根据截止时间约束、分配的作业槽数和作业可能的处理时间，提出新的作业序列以构建合理的任务列表。在生成的任务调度中，任务被调度到机架本地服务器、集群本地服务器和远程服务器的承诺槽位上，这极大地提高了数据局部性。在任务和槽之间分配之后，提出了对集群中可用槽的更新，不仅可以找到可用槽，而且可以根据当前CPU、内存和带宽利用率，利用可用槽数量的模糊逻辑来提高服务器资源利用率。实验结果表明，与时隙总数可变的现有算法相比，所提出的启发式方法的能耗更低。
Nowadays, energy costs are the most important factor in cloud computing. Therefore, the implementation of energy-aware task scheduling methods is of utmost importance. A task scheduling framework considering deadlines, data locality and resource utilization is proposed to save on energy costs in heterogeneous clusters. The framework consists of task list construction, task scheduling and slot list updating. In terms of deadline constraints, number of job slots allocated and possible processing times of jobs, a new job sequence is proposed to construct an reasonable task list. Tasks are scheduled to promising slots from their rack-local servers, cluster-local servers and remote servers in the produced task scheduling, which greatly improves data locality. After the assignment among tasks and slots, an update of available slots in clusters is proposed not only to find available slots but also to improve server resource utilization using fuzzy logic with the available number of slots according to current CPU, memory and bandwidth utilization. Experimental results show that the proposed heuristic results in lower energy consumption than the adapted existing algorithms with a variable total number of slots.