Scheduling Hadoop Jobs to Meet Deadlines
Scheduling Hadoop Jobs to Meet Deadlines
复制标题
DOI：
10.1109/cloudcom.2010.97
复制
发表时间：
2010-11
期刊：
影响因子：
--
通讯作者：
Kamal Kc;Kemafor Anyanwu
中科院分区：
文献类型：
--
作者：
Kamal Kc;Kemafor Anyanwu
诸如截止日期之类的用户约束是现有的基于云的数据处理环境（诸如Hadoop）没有考虑的重要要求。在当前的实现中，默认情况下，作业以FIFO顺序调度，并具有其他基于优先级的调度器的选项。在本文中，我们扩展了真实的时间集群调度的方法，占两个阶段的计算风格的MapReduce。我们根据用户指定的截止日期约束制定了调度作业的标准，并讨论了我们对Hadoop截止日期约束调度程序的实现和初步评估，该调度程序确保只有能够满足截止日期的作业才被调度执行。
User constraints such as deadlines are important requirements that are not considered by existing cloud-based data processing environments such as Hadoop. In the current implementation, jobs are scheduled in FIFO order by default with options for other priority based schedulers. In this paper, we extend real time cluster scheduling approach to account for the two-phase computation style of MapReduce. We develop criteria for scheduling jobs based on user specified deadline constraints and discuss our implementation and preliminary evaluation of a Deadline Constraint Scheduler for Hadoop which ensures that only jobs whose deadlines can be met are scheduled for execution.