A parallel spatial data analysis infrastructure for the cloud
A parallel spatial data analysis infrastructure for the cloud
复制标题
DOI：
10.1145/2525314.2525347
复制
发表时间：
2013-11
期刊：
影响因子：
--
通讯作者：
S. Ray;Bogdan Simion;Angela Demke Brown;Ryan Johnson
中科院分区：
文献类型：
--
作者：
S. Ray;Bogdan Simion;Angela Demke Brown;Ryan Johnson
空间数据分析应用正在从建筑信息管理、环境评估和医学成像等广泛的领域中涌现。耗时的计算几何算法使这些应用程序缓慢，即使是中等大小的数据集。与此同时，通过多核机器和云计算，可用的处理核心迅速扩展。这些趋势的汇合需要有效的空间查询处理的并行化。不幸的是，传统的并行空间数据库无法处理云中常见的性能异构性。我们介绍Niharika，一个并行的空间数据分析基础设施，利用所有可用的核心在异构集群。Niharika首先使用去聚类技术来创建平衡的空间分区。然后，Niharika使用动态负载平衡来适应空间数据集中的性能异构性和处理偏斜。我们使用Amazon EC2实例，使用三种负载平衡算法和两种不同的空间数据集（均来自TIGER）评估Niharika。Niharika适应了EC2节点中的性能异构性，从而实现了出色的加速（例如，在最好的情况下，在16个4核EC2节点上使用64个核，达到63.6X），并且性能优于无法适应的方法。
Spatial data analysis applications are emerging from a wide range of domains such as building information management, environmental assessments and medical imaging. Time-consuming computational geometry algorithms make these applications slow, even for medium-sized datasets. At the same time, there is a rapid expansion in available processing cores, through multicore machines and Cloud computing. The confluence of these trends demands effective parallelization of spatial query processing. Unfortunately, traditional parallel spatial databases are ill-equipped to deal with the performance heterogeneity that is common in the Cloud. We introduce Niharika, a parallel spatial data analysis infrastructure that exploits all available cores in a heterogeneous cluster. Niharika first uses a declustering technique that creates balanced spatial partitions. Then, Niharika adapts to performance heterogeneity and processing skew in the spatial dataset using dynamic load-balancing. We evaluate Niharika with three load-balancing algorithms and two different spatial datasets (both from TIGER) using Amazon EC2 instances. Niharika adapts to the performance heterogeneity in the EC2 nodes, thereby achieving excellent speedups (e.g., 63.6X using 64 cores on 16 4-core EC2 nodes, in the best case) and outperforming an approach that does not adapt.