Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments
Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments
复制标题
Re-Stream：大数据流计算环境中实时、节能的资源调度
DOI：
10.1016/j.ins.2015.03.027
复制
发表时间：
2015-10-20
影响因子：
8.1
通讯作者：
Li, Keqin
中科院分区：
文献类型：
--
作者：
Sun, Dawei;Zhang, Guangyan;Li, Keqin
为了在大数据流计算环境中实现高能效和低响应时间，需要建立一个高效节能的资源调度和优化框架。在本文中，我们提出了一个实时和节能的资源调度和优化框架，称为再流。首先，Re-Stream描绘了能量消耗、响应时间和资源利用率之间的数学关系，并获得了满足高能效和低响应时间的条件。其次，利用分布式流计算理论建立了数据流图模型，确定了数据流图中的关键路径。这种方法有助于计算给定数据流速度下数据流图的资源分配方案的能耗。第三，Re-Stream根据体系结构的要求，利用能量有效的启发式和关键路径调度机制来分配任务。通过重新分配数据流图的关键路径上的关键顶点来在线优化调度机制，以最小化响应时间和系统波动。此外，Re-Stream合并了非关键路径上的非关键顶点，以提高能量效率。我们评估Re-Stream来衡量大数据流计算环境的能源效率和响应时间。实验结果表明，Re-Stream能够有效提高大数据流计算系统的能量效率，缩短系统的平均响应时间。Re-Stream在大数据流计算环境中有效地提供了提高能源效率和减少响应时间之间的平衡。(C)2015 Elsevier Inc. All rights reserved.
To achieve high energy efficiency and low response time in big data stream computing environments, it is required to model an energy-efficient resource scheduling and optimization framework. In this paper, we propose a real-time and energy-efficient resource scheduling and optimization framework, termed the Re-Stream. Firstly, the Re-Stream profiles a mathematical relationship among energy consumption, response time, and resource utilization, and obtains the conditions to meet high energy efficiency and low response time. Secondly, a data stream graph is modeled by using the distributed stream computing theories, which identifies the critical path within the data stream graph. Such a methodology aids in calculating the energy consumption of a resource allocation scheme for a data stream graph at a given data stream speed. Thirdly, the Re-Stream allocates tasks by utilizing an energy-efficient heuristic and a critical path scheduling mechanism subject to the architectural requirements. This is done to optimize the scheduling mechanism online by reallocating the critical vertices on the critical path of a data stream graph to minimize the response time and system fluctuations. Moreover, the Re-Stream consolidates the non-critical vertices on the non-critical path so as to improve energy efficiency. We evaluate the Re-Stream to measure energy efficiency and response time for big data stream computing environments. The experimental results demonstrate that the Re-Stream has the ability to improve energy efficiency of a big data stream computing system, and to reduce average response time. The Re-Stream provides an elegant trade-off between increased energy efficiency and decreased response time effectively within big data stream computing environments. (C) 2015 Elsevier Inc. All rights reserved.