Lightweight Compression Techniques for Optimizing Complex Database Queries

用于优化复杂数据库查询的轻量级压缩技术

• 批准号: 255187874
• 负责人: Professor Dr.-Ing. Wolfgang Lehner
• 金额: --
• 依托单位: Professur für Datenbanken
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2014
• 资助国家: 德国
• 起止时间: 2013-12-31 至 2017-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/255187874?language=en
• 关键词: Lightweight; Compression; Techniques; Optimizing; Complex

Large data sets, which are managed by a database system, are usually stored in compressed form on external storage media. This approach saves capacity on external storage as well as reduces the data load time to main memory. However, during query processing in database systems all relevant data including intermediate results are kept and processed uncompressed in main memory. Thereby, main memory and processor caches in particular in many-core architecture are more and more the bottleneck. To tackle this problem, we investigate lightweight compression techniques for query processing to establish a balanced ratio between reduced data volume and additional expenses for compression. Our main idea is to compress each intermediate result during query processing to increase memory bandwidth and to reduce query processing time. For that purpose, we are going to develop intra as well as inter operator compression and integrate those techniques in the whole query processing. Moreover, the compression of intermediate results will be incorporated in query optimization using an appropriate cost model. From our point of view, a balanced lightweight compression is a trend-setting technique to reduce memory bottlenecks in upcoming many-core architectures and to facilitate scalable database applications.

由数据库系统管理的大型数据集通常以压缩形式存储在外部存储介质上。这种方法节省了外部存储的容量，并减少了将数据加载到主存储器的时间。然而，在数据库系统中的查询处理过程中，包括中间结果在内的所有相关数据都保存在主内存中，并且以未压缩的形式进行处理。因此，主存和处理器缓存，尤其是多核体系结构中的主存和处理器缓存成为越来越多的瓶颈。为了解决这个问题，我们研究了用于查询处理的轻量级压缩技术，以在减少的数据量和额外的压缩开销之间建立一个平衡的比率。我们的主要思想是在查询处理过程中对每个中间结果进行压缩，以增加内存带宽，减少查询处理时间。为此，我们将开发运算符内和运算符间压缩，并将这些技术集成到整个查询处理中。此外，中间结果的压缩将被合并到使用适当成本模型的查询优化中。在我们看来，平衡的轻量级压缩是一种引领趋势的技术，可以减少即将到来的多核体系结构中的内存瓶颈，并促进可扩展的数据库应用程序。

项目成果

From a Comprehensive Experimental Survey to a Cost-based Selection Strategy for Lightweight Integer Compression Algorithms

从综合实验调查到轻量级整数压缩算法的基于成本的选择策略

• DOI: 10.1145/3323991
• 发表时间: 2019
• 期刊: ACM Transactions on Database Systems (TODS)
• 作者: Patrick Damme;Annett Ungethüm;Juliana Hildebrandt;Dirk Habich;Wolfgang Lehner
• 通讯作者: Wolfgang Lehner

A Benchmark Framework for Data Compression Techniques

数据压缩技术的基准框架

• DOI: 10.1007/978-3-319-31409-9_6
• 发表时间: 2015
• 作者: Patrick Damme;Dirk Habich;Wolfgang Lehner
• 通讯作者: Wolfgang Lehner

Direct Transformation Techniques for Compressed Data: General Approach and Application Scenarios

压缩数据的直接转换技术：通用方法和应用场景

• DOI: 10.1007/978-3-319-23135-8_11
• 发表时间: 2015
• 作者: Patrick Damme;Dirk Habich;Wolfgang Lehner
• 通讯作者: Wolfgang Lehner

Lightweight Data Compression Algorithms: An Experimental Survey (Experiments and Analyses)

轻量级数据压缩算法：实验调查（实验和分析）

• DOI: 10.5441/002/edbt.2017.08
• 发表时间: 2017
• 作者: Patrick Damme;Dirk Habich;Juliana Hildebrandt;Wolfgang Lehner
• 通讯作者: Wolfgang Lehner

MorphStore - In-Memory Query Processing based on Morphing Compressed Intermediates LIVE

MorphStore - 基于变形压缩中间体的内存中查询处理 LIVE

• DOI: 10.1145/3299869.3320234
• 发表时间: 1920
• 期刊: Proceedings of the 2019 International Conference on Management of Data
• 作者: Dirk Habich;Patrick Damme;Annett Ungethüm;Johannes Pietrzyk;Alexander Krause;Juliana Hildebrandt;Wolfgang Lehner
• 通讯作者: Wolfgang Lehner