BIGDATA: F: Collaborative Research: Optimizing Log-Structured-Merge-Based Big Data Management Systems

BIGDATA：F：协作研究：优化基于日志结构合并的大数据管理系统

• 批准号: 1838222
• 负责人: Evangelos Christidis
• 金额: $ 135.81万
• 依托单位: University of California-Riverside
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1838222&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Optimizing; Log

Modern big data management systems support fast read and write operations based on the unique identifier (key) of a record. That is, they are fast when inserting key-value pairs, and given a key they quickly return the value associated with that key. To do so, most such systems rely on a Log-Structured-Merge Tree (LSM) structure that batches writes together before writing them to persistent storage. This project will study how to efficiently support more sophisticated operations on LSM-based storage systems, that is, operations that do not simply specify the key of a record. Examples of such operations include searching for records based instead on their location or time. By optimizing the storage and management of big data, this project has the potential to cut the storage costs and energy consumption in data centers. Further, the successful completion of this work will allow users to manage more data with the existing hardware infrastructure, which is critical given the new wave of big data being generated by sensors and the Internet-of-Things. The project will capitalize on the student diversity at two Hispanic Serving Institutions, and thus broaden the participation of under-represented groups in the research process.To support richer data modeling and querying capabilities on top of LSM key-value stores, this project will develop novel LSM indexing and access algorithms to support query plans that utilize both primary and secondary LSM components. In addition, it will design and evaluate flow control policies to dampen or eliminate the notoriously bursty data ingestion behavior that LSM-based storage structures exhibit. It will also study how to automatically and dynamically change LSM compaction policies and parameters based on the query workload. Data-semantics-aware compaction techniques will also be studied. The project will additionally develop novel LSM-aware query optimization techniques; the LSM storage layer is currently treated as a black box by most query optimizers. The planned methods will be deployed and evaluated on the open source Apache AsterixDB system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代大数据管理系统基于记录的唯一标识符（键）支持快速读取操作。也就是说，它们在插入键值对时很快，并且给出了一个键，它们很快返回与该键关联的值。为此，大多数这样的系统依赖于对数结构化的树（LSM）结构，该结构在编写持续存储之前将它们批量编写在一起。该项目将研究如何有效地支持基于LSM的存储系统的更复杂的操作，即不仅仅指定记录的钥匙的操作。此类操作的示例包括搜索基于其位置或时间的记录。通过优化大数据的存储和管理，该项目有可能降低数据中心的存储成本和能源消耗。此外，这项工作的成功完成将使用户可以使用现有的硬件基础架构来管理更多数据，这至关重要，鉴于传感器和图像互联网生成的新大数据浪潮至关重要。 The project will capitalize on the student diversity at two Hispanic Serving Institutions, and thus broaden the participation of under-represented groups in the research process.To support richer data modeling and querying capabilities on top of LSM key-value stores, this project will develop novel LSM indexing and access algorithms to support query plans that utilize both primary and secondary LSM components.此外，它将设计和评估流量控制策略，以抑制或消除基于LSM的存储结构所展示的臭名昭著的爆发数据摄入行为。它还将研究如何根据查询工作负载自动和动态更改LSM压实策略和参数。还将研究数据 - 语音感知的压实技术。该项目还将开发新颖的LSM感知查询优化技术； LSM存储层目前被大多数查询优化器视为黑匣子。计划的方法将在开源Apache AsterixDB系统上进行部署和评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的评估标准通过评估来支持的。

项目成果

SpiderWeb: A Spatial Data Generator on the Web

SpiderWeb：网络上的空间数据生成器

• DOI: 10.1145/3397536.3422351
• 发表时间: 2020
• 期刊: ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems
• 作者: Katiyar, Puloma;Vu, Tin;Migliorini, Sara;Belussi, Alberto;Eldawy, Ahmed
• 通讯作者: Eldawy, Ahmed

Spatial data generators

空间数据生成器

• 发表时间: 2022
• 期刊: Spatial Gems
• 作者: Vu, Tin;Migliorini, Sara;Eldawy, Ahmed;Belussi, Alberto
• 通讯作者: Belussi, Alberto

A Learned Query Optimizer for Spatial Join

用于空间连接的学习查询优化器

• DOI: 10.1145/3474717.3484217
• 发表时间: 2021
• 期刊: International Conference on Advances in Geographic Information Systems (SIGSPATIAL
• 作者: Vu, Tin;Belussi, Alberto;Migliorini, Sara;Eldawy, Ahmed
• 通讯作者: Eldawy, Ahmed

Raptor: Large Scale Analysis of Big Raster and Vector Data

• DOI: 10.14778/3352063.3352107
• 发表时间: 2019-08-01
• 期刊: PROCEEDINGS OF THE VLDB ENDOWMENT
• 影响因子: 2.5
• 作者: Singla, Samriddhi;Eldawy, Ahmed;Mokbel, Mohamed F.
• 通讯作者: Mokbel, Mohamed F.

Efficient local locking for massively multithreaded in-memory hash-based operators

• DOI: 10.1007/s00778-020-00642-5
• 发表时间: 2021-02
• 期刊: The VLDB Journal
• 作者: Bashar Romanous;Skyler Windh;Ildar Absalyamov;Prerna Budhkar;R. Halstead;W. Najjar;V. Tsotras