EAGER: Tadoop: A Dual-Purpose Framework Taming the Bipolarity of Storage and Communication for High-Performance Computing and Data Analytics

EAGER：Tadoop：一个双用途框架，克服存储和通信的两极性，实现高性能计算和数据分析

• 批准号: 1561041
• 负责人: Weikuan Yu
• 金额: $ 20.48万
• 依托单位: Florida State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-16 至 2018-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1561041&HistoricalAwards=false
• 关键词: EAGER; Tadoop; Dual; Purpose; Framework

High-performance computing (HPC) providers and applications need next-generation solutions to process big data from scientific simulations. Conventional HPC systems found in national laboratories and universities are constructed based on the compute-centric paradigm while enterprise big data analytics applications prefer a data-centric paradigm such as MapReduce. Distinct architectural differences between these two paradigms demand unconventional approaches. This project takes a radically different approach to investigate key architectural components in compute-centric and data-centric paradigms, designs a transformative dual-purpose framework called Tadoop that addresses their bipolarity issues in storage and communication management, and unifies them for both HPC and enterprise analytics applications. This high-risk Tadoop framework can enable a transformative data infrastructure for both HPC and data analytics applications and lead to broader impact in several aspects, such as demonstrating the transformation of existing HPC infrastructures into dual-purpose systems for computing and analytics, improving computer science curricula and instruction effectiveness, strengthening multidisciplinary data analytics research, releasing open-source software code, and transferring technologies for commercial service.

高性能计算(HPC)提供商和应用程序需要下一代解决方案来处理来自科学模拟的大数据。国家实验室和大学中的传统HPC系统是基于以计算为中心的范例构建的，而企业大数据分析应用程序更喜欢以数据为中心的范例，如MapReduce.这两个范例之间截然不同的体系结构差异要求采用非常规方法。该项目采用了一种截然不同的方法来研究以计算为中心和以数据为中心的范例中的关键体系结构组件，设计了名为Tadoop的变革性双用途框架，该框架解决了存储和通信管理中的两极问题，并将它们统一用于HPC和企业分析应用程序。这个高风险的Tadoop框架可以为高性能计算和数据分析应用程序带来变革性的数据基础设施，并在几个方面产生更广泛的影响，例如展示现有的高性能计算基础设施转变为计算和分析两用系统，改善计算机科学课程和教学有效性，加强多学科数据分析研究，发布开源软件代码，以及将技术转移为商业服务。

项目成果

SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication

• DOI: 10.1109/ccgrid.2018.00078
• 发表时间: 2018-05
• 期刊: 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
• 作者: Huansong Fu;Manjunath Gorentla Venkata;Shaeke Salman;N. Imam;Weikuan Yu

Efficient User-Level Storage Disaggregation for Deep Learning

• DOI: 10.1109/cluster.2019.8891023
• 发表时间: 2019-09
• 期刊: 2019 IEEE International Conference on Cluster Computing (CLUSTER)
• 作者: Yue Zhu;Weikuan Yu;Bing Jiao;K. Mohror;A. Moody;Fahim Chowdhury

Compression of Time Evolutionary Image Data through Predictive Deep Neural Networks

通过预测深度神经网络压缩时间演化图像数据

• 发表时间: 2021
• 期刊: Cloud and Grid Computing
• 作者: Roy, Rupak;Sato, Kento;Bhattachrya, Subhadeep;Fang, Xingang;Joti, Yasumasa;Hatsui, Takaki;Hiraki, Toshiyuki;Guo, Jian;Yu, Weikuan.
• 通讯作者: Yu, Weikuan.

Exploration of memory hybridization for RDD caching in Spark

• DOI: 10.1145/3315573.3329988
• 发表时间: 2019-06
• 期刊: Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management
• 作者: Md. Muhib Khan;Muhammad Ahad Ul Alam;Amit Kumar Nath;Weikuan Yu

Multivariate modeling and two-level scheduling of analytic queries

• DOI: 10.1016/j.parco.2019.01.006
• 发表时间: 2019-07
• 期刊: Parallel Comput.
• 作者: Zhuo Liu;Amit Kumar Nath;Xiaoning Ding;Huansong Fu;Md. Muhib Khan;Weikuan Yu