II-EN: Hadoop NextGen Infrastructure for Heterogeneous Approaches to Data-Intensive Computing

II-EN：用于数据密集型计算异构方法的 Hadoop NextGen 基础设施

• 批准号: 1405688
• 负责人: Jimmy Lin
• 金额: $ 49.99万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-01 至 2017-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405688&HistoricalAwards=false
• 关键词: II; EN; Hadoop; NextGen; Infrastructure

The ability for organizations to process enormous quantities of data and to extract insights from those data has revolutionized commerce and science. This phenomenon, known as "big data", is shaping the very fabric of our society. Our insatiable appetite for more data, and knowledge from the data, requires significant computational infrastructure for storage and analytical capabilities. Continued investments in infrastructure for academic researchers are vital from two perspectives: From the research perspective, the university's ability to help advance the state of the art in big data technologies is dependent on access to the right computational resources. From the educational perspective, the university's mission to train the next generation of scientists and engineers cannot be successfully accomplished without big data infrastructure that is becoming essential to their careers. The goal of this project is to provide computational resources to researchers at the University of Maryland to continuing envisioning the future of big data.The modern empirical approach to tackling many challenges in natural language processing, information retrieval, data mining, machine learning, and other related domains involves exploiting large amounts of data to learn statistical models that are able to capture characteristics of the problem. A necessary ingredient to this "big data" approach is scalable infrastructure that can distribute computations across a cluster of machines. Hadoop, the open-source implementation of MapReduce, has achieved widespread adoption as the de facto platform for data-intensive computing.Broadly speaking, MapReduce excels at large-scale content analysis in an offline, batch setting. However, this is not enough: we need a data-intensive computing platform that supports heterogeneous models of computation. Hadoop NextGen (aka YARN), provides exactly this: it allows a physical cluster to support a wide range of computational models via a generic resource allocation framework.This project supports the acquisition of a Hadoop NextGen cluster at the University of Maryland to support the following activities:1. To explore computational models beyond MapReduce, including batch/online tradeoffs in machine learning, real-time streaming computations, and graph processing.2. To sustain innovations in algorithms for content analysis as well as modeling implicit and latent relationships between heterogeneous content (text, images, graphs, etc.) at scale.3. To exploit novel hardware architectures for data-intensive computing (e.g., Graphics Processing Units and Solid State Drives).These resources will help the Laboratory for Computational Linguistics and Information Processing (CLIP) and collaborators at the University of Maryland sustain and enhance its successful record of innovation and the integration of research and education.

组织处理大量数据并从这些数据中提取见解的能力已经彻底改变了商业和科学。这种被称为“大数据”的现象正在塑造我们社会的结构。我们对更多数据和来自数据的知识的贪得无厌的胃口需要大量的计算基础设施来存储和分析功能。 从两个角度来看，对学术研究人员基础设施的持续投资至关重要：从研究角度来看，大学帮助推进大数据技术发展的能力取决于获得正确的计算资源。从教育的角度来看，如果没有对他们的职业生涯至关重要的大数据基础设施，大学培养下一代科学家和工程师的使命就无法成功完成。 该项目的目标是为马里兰州大学的研究人员提供计算资源，以继续设想大数据的未来。现代经验方法解决自然语言处理，信息检索，数据挖掘，机器学习和其他相关领域的许多挑战，涉及利用大量数据来学习能够捕捉问题特征的统计模型。这种“大数据”方法的一个必要组成部分是可扩展的基础设施，可以将计算分布在机器集群中。Hadoop是MapReduce的开源实现，作为数据密集型计算的事实上的平台，它已经获得了广泛的采用。从广义上讲，MapReduce在离线批量设置下的大规模内容分析方面表现出色。然而，这还不够：我们需要一个支持异构计算模型的数据密集型计算平台。Hadoop NextGen（又名YARN）正是提供了这一点：它允许物理集群通过通用的资源分配框架来支持广泛的计算模型。该项目支持马里兰州大学收购Hadoop NextGen集群，以支持以下活动：1.探索MapReduce之外的计算模型，包括 机器学习中的批处理/在线权衡，实时流 计算和图形处理。为了保持内容分析算法的创新， 作为建模异构之间的隐式和潜在的关系， 内容（文本、图像、图表等）按比例3.为了开发新的硬件架构， 计算（例如，这些资源将帮助马里兰州大学的计算语言学和信息处理实验室（CLIP）及其合作者维持和加强其创新和研究与教育一体化的成功记录。