SHF: Small: Hyperscaling Data Analytics for High-Performance Computers

SHF：小型：高性能计算机的超大规模数据分析

• 批准号: 1816577
• 负责人: Spyros Blanas
• 金额: $ 46万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816577&HistoricalAwards=false
• 关键词: SHF; Small; Hyperscaling; Data; Analytics

Data analytics extracts insights from massive datasets, often with the assistance of machine learning techniques. The goal of this project is to allow domain experts, including data scientists, to analyze massive datasets quickly using the most powerful supercomputing systems in the world. The problem is that state-of-the-art data processing algorithms that filter data, summarize results and combine information from different sources have inherent scalability bottlenecks. This project designs hyperscalable data processing algorithms that harness the unprecedented compute, storage and networking concurrency of a high-performance computer. This project also develops an open-source data processing engine to disseminate prototype implementations of these algorithms to the public. Another contribution is the creation of a massively parallel data processing module and associated teaching materials for undergraduate data science curricula, such as the diverse Data Analytics undergraduate major at The Ohio State University.The confluence of extreme compute parallelism, fast networking and growing memory capacities in the modern datacenter presents an opportunity to design a hyperscalable data processing kernel for warehouse-scale computers. This project sits at the intersection of data management and high-performance computing; it develops scalable join and aggregation algorithms, topology-conscious query planning and optimization techniques, and interference-aware data access methods for shared cold storage. This is accomplished by carefully overlapping communication and computation, identifying and avoiding unscalable all-to-all communication, accounting for network path congestion and variability in remote memory access latency, and judiciously using inter-process coordination to accelerate data ingestion from a massively parallel shared file system. These research activities lay the intellectual foundation to make data analytics scalable and efficient in warehouse-scale computers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据分析通常在机器学习技术的帮助下从海量数据集中提取见解。该项目的目标是让包括数据科学家在内的领域专家能够使用世界上最强大的超级计算系统快速分析海量数据集。问题在于，过滤数据、总结结果和组合来自不同来源的信息的最先进的数据处理算法具有固有的可扩展性瓶颈。该项目设计了超可扩展的数据处理算法，利用高性能计算机前所未有的计算、存储和网络并发性。该项目还开发了一个开源数据处理引擎，以向公众传播这些算法的原型实现。另一个贡献是为本科数据科学课程创建了大规模并行数据处理模块和相关教材，例如俄亥俄州立大学的多样化数据分析本科专业。现代数据中心的极端计算并行性、快速网络和不断增长的内存容量的融合提供了为仓库规模计算机设计超可扩展数据处理内核的机会。该项目位于数据管理和高性能计算的交叉点；它开发可扩展的连接和聚合算法、拓扑感知的查询规划和优化技术以及用于共享冷存储的干扰感知数据访问方法。这是通过仔细重叠通信和计算、识别和避免不可扩展的全对全通信、考虑网络路径拥塞和远程内存访问延迟的可变性以及明智地使用进程间协调来加速从大规模并行共享文件系统的数据摄取来实现的。这些研究活动奠定了知识基础，使数据分析在仓库规模的计算机中可扩展且高效。该奖项反映了 NSF 的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

ApproxJoin: Approximate Distributed Joins

• DOI: 10.1145/3267809.3267834
• 发表时间: 2018-10
• 期刊: Proceedings of the ACM Symposium on Cloud Computing
• 作者: D. Quoc;Istemi Ekin Akkus;Pramod Bhatotia;Spyros Blanas;Ruichuan Chen;C. Fetzer;T. Strufe

Beyond MPI: New Communication Interfaces for Database Systems and Data-Intensive Applications

超越 MPI：数据库系统和数据密集型应用程序的新通信接口

• DOI: 10.1145/3456859.3456862
• 发表时间: 2021
• 期刊: ACM SIGMOD Record
• 作者: Liu, Feilong;Barthels, Claude;Blanas, Spyros;Kimura, Hideaki;Swart, Garret
• 通讯作者: Swart, Garret

Characterizing I/O optimization opportunities for array-centric applications on HDFS

• DOI: 10.1109/hpec.2018.8547529
• 发表时间: 2018-09
• 期刊: 2018 IEEE High Performance extreme Computing Conference (HPEC)
• 作者: Donghe Kang;Vedang Patel;Kalyan Khandrika;Spyros Blanas;Yang Wang;S. Parthasarathy

Comparison of Array Management Library Performance - A Neuroscience Use Case

阵列管理库性能比较 - 神经科学用例

• 发表时间: 2019
• 期刊: and Analysis (Poster session
• 作者: Kang, Donghe;Rübel, Oliver;Byna, Suren;Blanas, Spyros
• 通讯作者: Blanas, Spyros

Design and Evaluation of an RDMA-aware Data Shuffling Operator for Parallel Database Systems

• DOI: 10.1145/3360900
• 发表时间: 2019-12
• 期刊: ACM Transactions on Database Systems (TODS)
• 作者: Feilong Liu;Lingyan Yin;Spyros Blanas