Efficient Join Synopsis Maintenance for Data Warehouse
Efficient Join Synopsis Maintenance for Data Warehouse
复制标题
数据仓库的高效连接概要维护
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Yuxi Liu
中科院分区：
文献类型：
--
作者：
Zhuoyue Zhao;Feifei Li;Yuxi Liu
日常业务运营和来自不同物联网应用的传感器等各种来源不断产生大量数据。它们通常被加载到数据仓库系统中，以执行复杂的分析。但是，如果查询涉及到连接，特别是多个大型表上的多对多连接，则代价可能非常高。对于直方图构建、模型训练等许多应用程序，连接摘要（即连接结果上的一个小的均匀随机样本）通常足以作为完整连接结果的代表性替代方案。为此，我们提出了一种新的算法SJoin，它可以在具有持续更新流入的动态数据库中对预先指定的一般θ-join查询维护连接摘要。SJoin的核心是维护一个加权连接图索引，这有助于在更新时有效地替换摘要中的连接结果。我们在几个复杂的连接查询上使用TPC-DS和模拟的道路传感器数据进行了广泛的实验，它们证明了SJoin优于最佳可用基线的明显优势。
Various sources such as daily business operations and sensors from different IoT applications constantly generate a lot of data. They are often loaded into a data warehouse system to perform complex analysis over. It, however, can be extremely costly if the query involves joins, especially many-to-many joins over multiple large tables. A join synopsis, i.e., a small uniform random sample over the join result, often suffices as a representative alternative to the full join result for many applications such as histogram construction, model training and etc. Towards that end, we propose a novel algorithm SJoin that can maintain a join synopsis over a pre-specified general θ-join query in a dynamic database with continuous inflows of updates. Central to SJoin is maintaining a weighted join graph index, which assists to efficiently replace join results in the synopsis upon update. We conduct extensive experiments using TPC-DS and a simulated road sensor data over several complex join queries and they demonstrate the clear advantage of SJoin over the best available baseline.