Using Simple PID Controllers to Prevent and Mitigate Faults in Scientific Workflows
Using Simple PID Controllers to Prevent and Mitigate Faults in Scientific Workflows
复制标题
使用简单的 PID 控制器预防和减轻科学工作流程中的故障
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
M. Atkinson
中科院分区：
文献类型：
--
作者：
Rafael Ferreira da Silva;Rosa Filgueira;E. Deelman;E. Pairo;I. Overton;M. Atkinson
科学工作流已经成为进行大规模科学研究的主流。因此，许多工作流应用程序和工作流管理系统（WMS）已被开发为网络基础设施的一部分，以允许科学家在一系列分布式平台上无缝执行其应用程序。尽管有许多成功的案例，但在分布式系统中运行工作流的一个关键挑战是故障预测、检测和恢复。在本文中，我们提出了一种方法，使用控制理论的自主计算的一部分，预测故障发生之前，并减轻他们在可能的情况下。所提出的方法应用比例-积分-微分控制器（PID控制器）的控制回路机制，这是广泛应用于工业控制系统，以减轻故障，通过调整控制器的输入。PID控制器旨在提前足够远地检测故障的可能性，以便可以执行动作以防止其发生。为了证明该方法的可行性，我们解决了大数据时代的两个常见的执行故障-数据存储过载和内存过载。我们定义、实现和评估简单的PID控制器，以自主管理生物信息学工作流程的数据和内存使用，该工作流程消耗/产生超过4.4TB的数据，并需要超过24 TB的内存来同时运行所有任务。实验结果表明，工作流执行可能会显着贝内于PID控制器，特别是在在线和未知条件下。仿真结果表明，接近最佳的执行（减速1.01），可以达到当使用我们提出的方法，并检测到故障，并减轻其发生的时间。
Scientific workflows have become mainstream for conducting large-scale scientific research. As a result, many workflow applications and Workflow Management Systems (WMSs) have been developed as part of the cyberinfrastructure to allow scientists to execute their applications seamlessly on a range of distributed platforms. In spite of many success stories, a key challenge for running workflows in distributed systems is failure prediction, detection, and recovery. In this paper, we propose an approach to use control theory developed as part of autonomic computing to predict failures before they happen, and mitigated them when possible. The proposed approach applying the proportional-integral-derivative controller (PID controller) control loop mechanism, which is widely used in industrial control systems, to mitigate faults by adjusting the inputs of the controller. The PID controller aims at detecting the possibility of a fault far enough in advance so that an action can be performed to prevent it from happening. To demonstrate the feasibility of the approach, we tackle two common execution faults of the Big Data era—data storage overload and memory overflow. We define, implement, and evaluate simple PID controllers to autonomously manage data and memory usage of a bioinformatics workflow that consumes/produces over 4.4TB of data, and requires over 24TB of memory to run all tasks concurrently. Experimental results indicate that workflow executions may significantly benefit from PID controllers, in particular under online and unknown conditions. Simulation results show that nearly-optimal executions (slowdown of 1.01) can be attained when using our proposed method, and faults are detected and mitigated far in advance of their occurence.