A Batch System with Efficient Adaptive Scheduling for Malleable and Evolving Applications
A Batch System with Efficient Adaptive Scheduling for Malleable and Evolving Applications
复制标题
DOI：
10.1109/ipdps.2015.34
复制
发表时间：
2015-05
期刊：
影响因子：
--
通讯作者：
Suraj Prabhakaran;M. Neumann;Sebastian Rinke;F. Wolf;Abhishek K. Gupta;L. Kalé
中科院分区：
文献类型：
--
作者：
Suraj Prabhakaran;M. Neumann;Sebastian Rinke;F. Wolf;Abhishek K. Gupta;L. Kalé
超级计算机的吞吐量不仅取决于高效的作业调度，还取决于构成工作负载的作业类型。可塑性工作对于集群来说是最有利的，因为它们可以动态地适应不断变化的资源分配。批处理系统可以扩展或收缩正在运行的可延展作业，以提高系统利用率、吞吐量和响应时间。然而，在过去，像 MPI 这样的常用编程模型的刚性使得编写可延展的应用程序成为一项艰巨的任务，这就是为什么它在很大程度上仍未实现。现在这种情况正在改变。为了提高新兴精确系统的容错能力、负载不平衡和能源效率，诸如 Charm++ 等更具适应性的编程范例应运而生。尽管它们为可延展性提供了更好的支持，但当前的批处理系统仍然缺乏可延展性作业的管理设施，因此无法发挥其潜力。在本文中，我们提出了 Torque/Maui 批处理系统的延展性扩展。我们提出了一种新颖的可延展作业调度策略，并展示了第一个能够有效管理刚性、可延展和不断发展的作业的批处理系统。我们证明，与其他最先进的可延展作业调度策略相比，我们的策略在不同的工作负载动态下始终实现了卓越的性能。
The throughput of supercomputers depends not only on efficient job scheduling but also on the type of jobs that form the workload. Malleable jobs are most favourable for a cluster as they can dynamically adapt to a changing allocation of resources. The batch system can expand or shrink a running malleable job to improve system utilization, throughput, and response times. In the past, however, the rigid nature of commonly used programming models like MPI made writing malleable applications a daunting task, which is why it remained largely unrealized. This is now changing. To improve fault tolerance, load imbalance, and energy efficiency in emerging exactable systems, more adaptive programming paradigms such as Charm++ enter the scene. Although they offer better support for malleability, current batch systems still lack management facilities for malleable jobs and are therefore incapable of leveraging their potential. In this paper, we present an extension of the Torque/Maui batch system for malleability. We propose a novel malleable job scheduling strategy and show the first batch system capable of efficiently managing rigid, malleable, and evolving jobs together. We demonstrate that our strategy achieves consistently superior performance in comparison to every other state-of-the-art malleable job scheduling strategy under varying dynamics of the workload.