A containerized task clustering for scheduling workflows to utilize processors and containers on clouds
A containerized task clustering for scheduling workflows to utilize processors and containers on clouds
复制标题
DOI：
10.1007/s11227-021-03789-2
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
H. Kanemitsu;Kenji Kanai;J. Katto;H. Nakazato
中科院分区：
文献类型：
--
作者：
H. Kanemitsu;Kenji Kanai;J. Katto;H. Nakazato
用于并行处理的虚拟化技术的最新进展涉及在工作流中调度容器化任务。由于容器可以包含多个任务，因此可以在应用程序之间重用或共享。如果工作流中的每个任务都使用其专用容器，而没有在任何任务之间共享，则必须为每个任务下载每个容器映像。因此，需要许多计算资源来处理，并且与容器映像下载相关的通信延迟可能成为完工时间的瓶颈。在工作流的任务调度算法中，如何有效地在任务间共享容器，避免冗余的容器镜像下载过程和冗余的任务分配是一个新的挑战性问题。其中一个根本问题是，没有政策已经建立了同时满足有效的容器共享，保持任务的并行度，有效的计算资源利用率。在本文中，我们提出了一个基于集群的云容器化任务调度算法，即共享功能任务集群利用虚拟化资源（SF-CUV）。SF-CUV的目标是通过聚类任务和共享任务间的容器，以较少的计算资源和容器比其他算法最大限度地减少完工时间。SF-CUV包括两个阶段：（i）任务聚类和预虚拟CPU（vCPU）分配阶段，以获得准确的调度优先级，以及（ii）任务排序和实际任务重新分配阶段。通过仿真和在真实的环境中获得的实验结果表明，SF-CUV可以同时利用vCPU和容器，与其他方法相比，具有更短的完工时间。
Recent advancements of virtualization technologies for parallel processing involve scheduling containerized tasks in a workflow. Since a container can include multiple tasks, it can be reused or shared among applications. If every task in a workflow uses its dedicated container without sharing among any tasks, each container image must be downloaded for each task. As a result, many computational resources are required to process and the communication latency related to container image downloading can become a bottleneck for the makespan. In task scheduling algorithms for workflows, this characteristic produces a new challenging issue that how effectively shares containers among tasks to avoid redundant container image download processes and redundant task allocations. One of the fundamental problems is that no policy has been established for simultaneously satisfying effective container sharing, maintaining the degree of task parallelism, and effective computational resource utilization. In this paper, we propose a clustering-based containerized task scheduling algorithm for clouds, namely, shareable functional task clustering for utilizing virtualized resources (SF-CUV). The objective of SF-CUV is to minimize the makespan with less computational resources and containers than other algorithms by clustering tasks and sharing each container among tasks. SF-CUV consists of two phases: (i)task clustering and pre-virtual CPU (vCPU) allocation phase to derive an accurate scheduling priority, and (ii)task ordering and actual task reallocation phase. Experimental results obtained via simulation and in a real environment show that SF-CUV can utilize both vCPUs and containers with a shorter makespan compared with other approaches.