Monitoring Hadoop by Using IEEE1888 in Implementing Energy-Aware Thread Scheduling
Monitoring Hadoop by Using IEEE1888 in Implementing Energy-Aware Thread Scheduling
复制标题
DOI：
10.1109/uic-atc-scalcom.2014.55
复制
发表时间：
2014-12
期刊：
影响因子：
--
通讯作者：
Hiroaki Takasaki;S. Mostafa;S. Kusakabe
中科院分区：
文献类型：
--
作者：
Hiroaki Takasaki;S. Mostafa;S. Kusakabe
Hadoop是一个软件框架，用于编写通常在商用计算节点集群上并行处理大量数据的应用程序。在实现我们的能量感知调度框架的Hadoop系统上的商品节点组成，我们使用IEEE 1888，一个开放的网络协议，无处不在的绿色社区控制网络协议。我们的调度框架最初是为单节点商品平台开发的，由内核级线程调度器、API、监视器和调度器行为控制器组成。为了在Hadoop集群上实现该框架，我们使用IEEE1888对网络上的从节点进行监控。我们的框架通过API监控性能指标并调整调度器的行为。该框架可以有效地减少内存密集型工作负载的线程启用Hadoop的能源消耗。
Hadoop is a software framework for writing applications that process huge amount of data in parallel typically on clusters of commodity computing nodes. In implementing our energy-aware scheduling framework on a Hadoop system consisting of commodity nodes, we use IEEE1888, an open network protocol, Ubiquitous Green Community Control Network Protocol. Our scheduling framework is originally developed for single-node commodity platforms and consists of a kernel level thread scheduler, API, a monitor and a controller for the behavior of the scheduler. In order to implement this framework on a Hadoop cluster, we monitor and control the slave nodes over the network by using IEEE1888. Our framework monitors performance metrics and adjusts the behavior of the scheduler through the API. The framework can effectively reduce energy consumption for memory-intensive workload on thread-enabled Hadoop.