Lightweight Compression Techniques for Optimizing Complex Database Queries

由数据库系统管理的大型数据集通常以压缩形式存储在外部存储介质上。这种方法节省了外部存储器的容量，并减少了数据加载到主存储器的时间。然而，在数据库系统中的查询处理期间，所有相关数据（包括中间结果）在主存储器中被保持和处理为未压缩的。因此，主存储器和处理器高速缓存，特别是在众核架构中，越来越成为瓶颈。为了解决这个问题，我们调查轻量级压缩技术的查询处理，以建立一个平衡的比例之间减少数据量和额外的费用压缩。我们的主要思想是压缩查询处理过程中的每个中间结果，以增加内存带宽，减少查询处理时间。为此，我们将开发内部以及内部运算符压缩，并将这些技术集成到整个查询处理中。此外，中间结果的压缩将被纳入查询优化使用适当的成本模型。从我们的观点来看，平衡的轻量级压缩是一种趋势设定技术，可以减少即将到来的众核架构中的内存瓶颈，并促进可伸缩的数据库应用程序。

Large data sets, which are managed by a database system, are usually stored in compressed form on external storage media. This approach saves capacity on external storage as well as reduces the data load time to main memory. However, during query processing in database systems all relevant data including intermediate results are kept and processed uncompressed in main memory. Thereby, main memory and processor caches in particular in many-core architecture are more and more the bottleneck. To tackle this problem, we investigate lightweight compression techniques for query processing to establish a balanced ratio between reduced data volume and additional expenses for compression. Our main idea is to compress each intermediate result during query processing to increase memory bandwidth and to reduce query processing time. For that purpose, we are going to develop intra as well as inter operator compression and integrate those techniques in the whole query processing. Moreover, the compression of intermediate results will be incorporated in query optimization using an appropriate cost model. From our point of view, a balanced lightweight compression is a trend-setting technique to reduce memory bottlenecks in upcoming many-core architectures and to facilitate scalable database applications.