Fault-tolerant scheduling and data placement for scientific workflow processing in geo-distributed clouds
Fault-tolerant scheduling and data placement for scientific workflow processing in geo-distributed clouds
复制标题
地理分布式云中科学工作流程处理的容错调度和数据放置
DOI：
10.1016/j.jss.2022.111227
复制
发表时间：
2022-02-02
影响因子：
3.5
通讯作者：
Luo, Youlong
中科院分区：
文献类型：
--
作者：
Li, Chunlin;Liu, Jun;Luo, Youlong
在地理分布式云环境下，科学的工作流处理对于大规模任务的调度和任务间海量数据的放置至关重要。然而，在不同地理分布的数据中心中处理科学工作流时，任务执行时间和数据传输能耗是两个迫切需要解决的问题。针对数据放置问题，提出了一种拉格朗日松弛法。该方法从工作负载均衡、存储容量、数据依赖性、传输带宽、传输成本等方面综合考虑，获取数据传输时间最小。针对任务调度问题，提出了一种容错调度策略。该策略通过考虑任务执行时间和能量消耗来优化任务调度机制。在数据放置方面，实验结果表明，与ILP-FDP算法、GA-DPSO算法和GPDP算法相比，所提松弛算法的数据传输时间平均可分别减少14.61%、38.03%和39.57%。在容错调度方面，TSPT算法的能耗最低。与MTS算法、EODS算法和EWTS算法相比，本文算法的平均增益分别为15.33%、16.65%和28.96%。与基准算法相比，TSPT算法的任务执行时间平均可分别减少12.78、18.85和25.65。(C),2022爱思唯尔公司版权所有。
Scientific workflow processing in geo-distributed cloud is crucial for the scheduling of large-scale tasks and the massive data placement among tasks. However, the task execution time and energy consumption of data transmission are two urgent issues when a scientific workflow is processed in the different geo-distributed data centers. Aiming at the data placement problem, this paper proposes a Lagrangian relaxation method. This method considers the workload balance, storage capacity, data dependency, transmission bandwidth, and transmission cost to obtain the minimum time of data transmission time. Aiming at the task scheduling problems, a fault-tolerant scheduling strategy is proposed. The strategy optimizes the task scheduling mechanism by considering the task execution time and energy consumption. Finally, the performance of the proposed methods is evaluated via extensive experiments In terms of the data placement, the experiment results imply that the data transmission time of the proposed relaxation algorithm can averagely achieve up to 14.61%, 38.03%, and 39.57% reduction of over that of ILP-FDP algorithm, GA-DPSO algorithm, and GPDP algorithm, respectively. As for the fault-tolerant scheduling, the energy consumption of the TSPT algorithm is the lowest. Compared with the MTS algorithm, EODS algorithm and EWTS algorithm, the average gains of the proposed algorithm are 15.33%, 16.65%, and 28.96%, respectively. Compared with the benchmark algorithms, the task execution time of the TSPT algorithm can averagely reduce up to 12.78,18.85 and 25.65, respectively. (C)& nbsp;2022 Elsevier Inc. All rights reserved.