Context-aware task allocation for fast parallel big data processing in optical-wireless networks
Context-aware task allocation for fast parallel big data processing in optical-wireless networks
复制标题
DOI：
10.1109/iwcmc.2014.6906394
复制
发表时间：
2014-09
期刊：
影响因子：
--
通讯作者：
Katsuya Suto;Hiroki Nishiyama;N. Kato
中科院分区：
文献类型：
--
作者：
Katsuya Suto;Hiroki Nishiyama;N. Kato
MapReduce体系结构被认为是高效、可靠的大数据挖掘最有前途的候选者之一。目前的MapReduce基本上是为数据中心和企业网络而设计的，其中多个服务器通过光缆互连，未来的MapReduce将应用于光无线环境中，如光无线数据中心网络、光纤无线(FiWi)接入网络等。要将MapReduce修改为适用于光无线混合网络，我们需要回答一个基础研究问题，即MapReduce体系结构如何利用光和无线资源进行任务分配？为了回答这个问题，本文揭示了一些具有挑战性的问题，并结合光通信和无线通信的特点，提出了一种上下文感知的任务分配方案。我们提出的任务分配方案可以最大限度地减少大数据处理的完成时间。数值结果表明，与现有的任务分配方案相比，本文提出的方法是有效的。
MapReduce architecture has been considered as one of the most promising candidates for efficient and reliable big data mining. While current MapReduce is basically designed for data center and enterprise networks, in which a number of servers are interconnected with optical fiber cables, prospective MapReduce would be applied in optical-wireless environment such as optical-wireless data center network, fiber-wireless (FiWi) access network, and so forth. To modify MapReduce for opticalwireless hybrid network, we need to answer the fundamental research problem, “How does MapReduce architecture use optical and wireless resources for task allocation?” To answer this question, this paper reveals some challenging issues and proposes a context-aware task allocation scheme that is designed by considering characteristics of both optical and wireless communications. Our proposed task allocation scheme can minimize the completion time of big data processing. Numerical results are presented to demonstrate the effectiveness of our proposed method compared with existing task allocation schemes.