Partition-Based Online Aggregation with Shared Sampling in the Cloud
Partition-Based Online Aggregation with Shared Sampling in the Cloud
复制标题
基于分区的在线聚合以及云中的共享采样
DOI：
10.1007/s11390-013-1393-6
复制
发表时间：
2013-11
影响因子：
0.7
通讯作者：
东方
中科院分区：
文献类型：
--
作者：
王宇翔;罗军舟;宋爱波;东方
在线聚合是一种有吸引力的基于采样的技术，它通过对最终结果的估计来响应聚合查询，并且置信区间会随着时间变得更窄。它已被构建到一个用于大数据的基于MapReduce的云系统中。
Online aggregation is an attractive sampling-based technology to response aggregation queries by an estimate to the final result, with the confidence interval becoming tighter over time. It has been built into a MapReduce-based cloud system for big data analytics, which allows users to monitor the query progress, and save money by killing the computation early once sufficient accuracy has been obtained. However, there are several limitations that restrict the performance of online aggregation generated from the gap between the current mechanism of MapReduce paradigm and the requirements of online aggregation, such as: 1) the low sampling efficiency due to the lack of consideration of skewed data distribution for online aggregation in MapReduce, and 2) the large redundant I/O cost of online aggregation caused by the independent job execution mechanism of MapReduce. In this paper, we present OLACloud, a MapReduce-based cloud system to well support online aggregation for different data distributions and large-scale concurrent query processing. We propose a content-aware repartition method with a fair-allocation block placement strategy to increase the sampling efficiency and guarantee the storage and computation load balancing simultaneously. We also develop a shared sampling method to share the sampling opportunities among multiple queries to reduce redundant I/O cost. We also implement OLACloud in Hadoop, and conduct an extensive experimental study on the TPC-H benchmark for skewed data distribution. Our results demonstrate the efficiency and effectiveness of OLACloud.
登录
查看更多内容
DOI：
10.1145/564691.564721
发表时间：
2002-06
期刊：
--
影响因子：
--
作者：
Gang Luo;Curt J. Ellmann;P. Haas;J. Naughton
通讯作者：
Gang Luo;Curt J. Ellmann;P. Haas;J. Naughton
DOI：
10.1007/978-3-319-32025-0
发表时间：
2016
期刊：
--
影响因子：
--
作者：
Hong Gao;Jinho Kim;Yasushi Sakurai
通讯作者：
Hong Gao;Jinho Kim;Yasushi Sakurai
DOI：
10.1007/978-3-319-05813-9
发表时间：
2014
期刊：
--
影响因子：
--
作者：
通讯作者：
--
DOI：
--
发表时间：
2011
期刊：
--
影响因子：
--
作者：
H. Herodotou;Harold Lim;Gang Luo;N. Borisov;Liang Dong;Fatma Bilgen Cetin;S. Babu
通讯作者：
H. Herodotou;Harold Lim;Gang Luo;N. Borisov;Liang Dong;Fatma Bilgen Cetin;S. Babu
DOI：
10.1109/ssdm.1997.621151
发表时间：
1997-08
期刊：
Proceedings. Ninth International Conference on Scientific and Statistical Database Management (Cat. No.97TB100150)
影响因子：
--
作者：
P. Haas
通讯作者：
P. Haas