SHF: Small: Collaborative Research: Modeling and Analyzing Big Data on Peta- and Exascale Distributed Systems Supported by MapReduce Methodologies

SHF：小型：协作研究：在 MapReduce 方法支持下对 Peta 和 Exascale 分布式系统上的大数据进行建模和分析

• 批准号: 1318417
• 负责人: Pietro Cicotti
• 金额: $ 6.9万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1318417&HistoricalAwards=false
• 关键词: SHF; Small; Collaborative; Research; Modeling

Current petascale platforms can perform large-scale simulations and generate massive amounts of data at unprecedented rates. These rates are expected to increase as exascale platforms are introduced. The generation of more and more data presents new challenges for scientists who struggle with the analysis, sorting, and selection of scientifically meaningful results. When very large amounts of data records are located across a large number of nodes in a distributed memory system, even a small number of comparisons can be costly or even impossible. Therefore, new methodologies are necessary to analyze large scientific datasets at scale.The goal of this project is to develop a transformative analysis method to model the properties of large scientific datasets in a distributed manner on petascale systems today and exascale systems in the future. The research activity includes (1) the design of new algorithms for encoding properties embedded in distributed data in a parallel manner by using space reduction techniques; (2) the design of new algorithms for clustering and classifying these properties by using distributed paradigms such as MapReduce; (3) the deployment of the algorithms for diverse datasets in structural biology and astronomy; and (4) the tuning of the algorithms for both result performance and accuracy on emerging storage technologies. The analysis method will provide the scientific community with infrastructures and instrumentations to identify features that can be used to predict class memberships; find recurrent patterns in datasets; and identify class memberships from a specific feature or property. By effectively and accurately capturing scientific information in a scalable manner, these infrastructures and instrumentations will break the traditional constraint of data centralization and allow scientists to overcome the difficulties associated with the fully distributed nature of the data considered.The project's educational component promotes training and learning in computational modeling and analysis techniques as well as data-intensive algorithms and platforms by involving undergraduate and graduate students in research activities and integrating big data analytics into the undergraduate curriculum at the University of Delaware. The research-based educational materials developed in this project will be made available to the scientific community through the project portal and through tutorials at XSEDE and Supercomputing (SC) conferences.

当前的千万级平台可以执行大规模模拟并以前所未有的速度生成大量数据。随着百亿亿级平台的推出，这些速率预计将会增加。越来越多的数据的产生给那些努力分析、排序和选择具有科学意义的结果的科学家带来了新的挑战。当大量数据记录分布在分布式内存系统中的大量节点上时，即使是少量的比较也可能代价高昂甚至不可能。因此，需要新的方法来大规模分析大型科学数据集。该项目的目标是开发一种变革性分析方法，以分布式方式在当今的千万级系统和未来的百亿亿级系统上对大型科学数据集的属性进行建模。研究活动包括（1）设计新算法，利用空间缩减技术以并行方式对嵌入分布式数据的属性进行编码； （2）使用MapReduce等分布式范式设计对这些属性进行聚类和分类的新算法； (3) 结构生物学和天文学中不同数据集的算法部署； (4) 针对新兴存储技术的结果性能和准确性调整算法。该分析方法将为科学界提供基础设施和仪器来识别可用于预测类别成员资格的特征；查找数据集中的重复模式；并根据特定功能或属性识别类成员资格。通过以可扩展的方式有效、准确地捕获科学信息，这些基础设施和仪器将打破数据集中的传统约束，使科学家能够克服与所考虑的数据的完全分布式性质相关的困难。该项目的教育部分通过让本科生和研究生参与研究活动并将大数据分析集成到计算模型和分析技术以及数据密集型算法和平台方面促进培训和学习。 特拉华大学的本科课程。该项目开发的基于研究的教育材料将通过项目门户以及 XSEDE 和超级计算 (SC) 会议上的教程向科学界提供。