Identifying Quick Starters: Towards an Integrated Framework for Efficient Predictions of Queue Waiting Times of Batch Parallel Jobs
Identifying Quick Starters: Towards an Integrated Framework for Efficient Predictions of Queue Waiting Times of Batch Parallel Jobs
复制标题
识别快速入门：建立一个有效预测批并行作业队列等待时间的集成框架
DOI：
--
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
Sathish S. Vadhiyar
中科院分区：
文献类型：
--
作者：
Rajath Kumar;Sathish S. Vadhiyar
生产并行系统是空间共享的，因此采用批处理队列，其中提交给系统的作业在执行前等待。因此，提交给并行批处理系统的作业除了执行时间外，还需要排队等待时间。这些队列等待时间的预测对于向用户提供总体估计是重要的，并且还可以帮助元调度器做出调度决策。对超级计算机作业轨迹的分析表明，大约56%到99%的作业的排队等待时间不到一个小时。因此，识别这些快速启动器或短队列等待时间的工作是必不可少的队列等待时间预测的整体改善。现有的策略提供了高度高估的队列等待时间的上限，使边界不太有用的短队列等待时间的作业。在这项工作中，我们已经开发了一个综合框架，使用的工作特性，状态的队列和处理器占用率，以确定和预测快速启动器，并使用现有的策略来预测长队列等待时间的工作。我们对不同生产超级计算机作业轨迹的实验表明，我们的预测策略可以正确识别多达20倍的快速启动者，并为这些作业提供更严格的界限，从而使整体预测准确率比现有方法高出64%。
Production parallel systems are space-shared and hence employ batch queues in which the jobs submitted to the systems are made to wait before execution. Thus, jobs submitted to parallel batch systems incur queue waiting times in addition to the execution times. Prediction of these queue waiting times is important to provide overall estimates to the users and can also help metaschedulers make scheduling decisions. Analyses of the job traces of supercomputers reveal that about 56 to 99% of the jobs incur queue waiting times of less than an hour. Hence, identifying these quick starters or jobs with short queue waiting times is essential for overall improvement on queue waiting time predictions. Existing strategies provide high overestimates of upper bounds of queue waiting times rendering the bounds less useful for jobs with short queue waiting times. In this work, we have developed an integrated framework that uses the job characteristics, and states of the queue and processor occupancy to identify and predict quick starters, and use the existing strategies to predict jobs with long queue waiting times. Our experiments with different production supercomputer job traces show that our prediction strategies can lead to correct identification of up to 20 times more quick starters and provide tighter bounds for these jobs, and thus result in up to 64% higher overall prediction accuracy than existing methods.