II-EN: Hadoop NextGen Infrastructure for Heterogeneous Approaches to Data-Intensive Computing

II-EN：用于数据密集型计算异构方法的 Hadoop NextGen 基础设施

• 批准号: 1405688
• 负责人: Jimmy Lin
• 金额: $ 49.99万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-01 至 2017-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405688&HistoricalAwards=false
• 关键词: II; EN; Hadoop; NextGen; Infrastructure

The ability for organizations to process enormous quantities of data and to extract insights from those data has revolutionized commerce and science. This phenomenon, known as "big data", is shaping the very fabric of our society. Our insatiable appetite for more data, and knowledge from the data, requires significant computational infrastructure for storage and analytical capabilities. Continued investments in infrastructure for academic researchers are vital from two perspectives: From the research perspective, the university's ability to help advance the state of the art in big data technologies is dependent on access to the right computational resources. From the educational perspective, the university's mission to train the next generation of scientists and engineers cannot be successfully accomplished without big data infrastructure that is becoming essential to their careers. The goal of this project is to provide computational resources to researchers at the University of Maryland to continuing envisioning the future of big data.The modern empirical approach to tackling many challenges in natural language processing, information retrieval, data mining, machine learning, and other related domains involves exploiting large amounts of data to learn statistical models that are able to capture characteristics of the problem. A necessary ingredient to this "big data" approach is scalable infrastructure that can distribute computations across a cluster of machines. Hadoop, the open-source implementation of MapReduce, has achieved widespread adoption as the de facto platform for data-intensive computing.Broadly speaking, MapReduce excels at large-scale content analysis in an offline, batch setting. However, this is not enough: we need a data-intensive computing platform that supports heterogeneous models of computation. Hadoop NextGen (aka YARN), provides exactly this: it allows a physical cluster to support a wide range of computational models via a generic resource allocation framework.This project supports the acquisition of a Hadoop NextGen cluster at the University of Maryland to support the following activities:1. To explore computational models beyond MapReduce, including batch/online tradeoffs in machine learning, real-time streaming computations, and graph processing.2. To sustain innovations in algorithms for content analysis as well as modeling implicit and latent relationships between heterogeneous content (text, images, graphs, etc.) at scale.3. To exploit novel hardware architectures for data-intensive computing (e.g., Graphics Processing Units and Solid State Drives).These resources will help the Laboratory for Computational Linguistics and Information Processing (CLIP) and collaborators at the University of Maryland sustain and enhance its successful record of innovation and the integration of research and education.

组织处理大量数据并从这些数据中提取见解的能力已经彻底改变了商业和科学。这种被称为“大数据”的现象正在塑造我们社会的结构。我们对更多数据和从数据中获取知识的贪得无厌的欲望，需要重要的计算基础设施来存储和分析能力。从两个角度来看，对学术研究人员基础设施的持续投资至关重要：从研究的角度来看，大学帮助推动大数据技术发展的能力依赖于获得正确的计算资源。从教育的角度来看，如果没有大数据基础设施，大学培养下一代科学家和工程师的使命就无法成功完成，而大数据基础设施对他们的职业生涯至关重要。该项目的目标是为马里兰大学的研究人员提供计算资源，以继续展望大数据的未来。解决自然语言处理、信息检索、数据挖掘、机器学习和其他相关领域的许多挑战的现代经验方法涉及利用大量数据来学习能够捕捉问题特征的统计模型。这种“大数据”方法的必要组成部分是可扩展的基础设施，它可以跨机器集群分发计算。Hadoop是MapReduce的开源实现，作为数据密集型计算的事实上的平台已经被广泛采用。一般来说，MapReduce擅长于离线、批处理设置下的大规模内容分析。然而，这还不够：我们需要一个支持异构计算模型的数据密集型计算平台。Hadoop NextGen（又名YARN）恰恰提供了这一点：它允许物理集群通过通用的资源分配框架来支持广泛的计算模型。该项目支持收购马里兰大学的Hadoop NextGen集群，以支持以下活动：探索MapReduce之外的计算模型，包括机器学习中的批处理/在线权衡，实时流计算和图处理。维持内容分析算法的创新，以及对大规模异构内容（文本、图像、图形等）之间隐含和潜在关系的建模。为数据密集型计算开发新的硬件架构（例如，图形处理单元和固态驱动器）。这些资源将帮助计算语言学和信息处理实验室（CLIP）和马里兰大学的合作者维持和提高其创新和研究与教育整合的成功记录。