BIGDATA: Small: DCM: Open Flow Enabled Hadoop over Local and Wide Area Clusters

BIGDATA：小型：DCM：本地和广域集群上支持开放流的 Hadoop

• 批准号: 1251201
• 负责人: Robert Grossman
• 金额: $ 74.98万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-06-01 至 2017-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1251201&HistoricalAwards=false
• 关键词: BIGDATA; Small; DCM; Open; Flow

BigData: Small: DCM: Open Flow Enabled Hadoop over Local and Wide Area ClustersRobert L. GrossmanUniversity of ChicagoIn the recent years, data intensive programming using Hadoop and MapReduce has become more and more important. As normally deployed, Hadoop's implementation of MapReduce in a multi-rack cluster is dependent upon the top of the rack switches and of the aggregator switches connecting multiple racks. To use Hadoop effectively at scale across many racks requires expensive network switches and routers that are complex to configure and to maintain.Software defined networks using OpenFlow have proven in many cases to offer good performance at lower cost and to be simpler to manage. The first goal of this proposal is to contribute to the development of a new version of Hadoop called Hadoop-OFP. The basic idea of Hadoop-OFP is to integrate OpenFlow enabled switches with Hadoop: to i) improve performance; ii) lower the cost of the hardware required; and iii) simplify the management of the cluster. Large data flows are also a critical component of data intensive computing. Unfortunately, setting up networks to manage large data flows can be challenging. A second goal of this proposal is to develop a tool that can configure OpenFlow enabled networks to handle more efficiently the large data flows that arise with data intensive computing.

BigData: Small: DCM: Open Flow Enabled Hadoop over Local and Wide Area ClustersRobert L. grossman芝加哥大学近年来，使用Hadoop和MapReduce进行数据密集型编程变得越来越重要。通常情况下，Hadoop在多机架集群中的MapReduce实现依赖于机架交换机的顶部和连接多个机架的聚合器交换机的顶部。要在多个机架上有效地大规模使用Hadoop，需要昂贵的网络交换机和路由器，这些交换机和路由器的配置和维护都很复杂。在许多情况下，使用OpenFlow的软件定义网络已被证明以较低的成本提供良好的性能，并且更易于管理。本提案的第一个目标是为Hadoop新版本Hadoop- ofp的开发做出贡献。Hadoop- ofp的基本思想是将启用OpenFlow的交换机与Hadoop集成在一起：1)提高性能；Ii)降低所需硬件的成本；iii)简化集群的管理。大数据流也是数据密集型计算的关键组成部分。不幸的是，建立管理大型数据流的网络可能具有挑战性。该提案的第二个目标是开发一种工具，可以配置OpenFlow支持的网络，以更有效地处理由数据密集型计算产生的大数据流。