ITISS: an efficient framework for querying big temporal data
ITISS: an efficient framework for querying big temporal data
复制标题
DOI：
10.1007/s10707-019-00362-1
复制
发表时间：
2019-05
期刊：
影响因子：
2
通讯作者：
Zhongpu Chen;Bin Yao;Zhi-Jie Wang;Wei Zhang-;Kai Zheng;Panos Kalnis;F. Tang
中科院分区：
文献类型：
--
作者：
Zhongpu Chen;Bin Yao;Zhi-Jie Wang;Wei Zhang-;Kai Zheng;Panos Kalnis;F. Tang
在真实的世界中，时态数据广泛存在于各种应用中，并且在不断增长。大时态数据的海量性和实时响应的要求使得对大时态数据进行有效的管理和操作成为一个迫切的问题和挑战。由于单机系统的计算能力有限，采用分布式系统处理大规模的时态数据是一种理想的选择。然而，现有的分布式系统或方法要么是基于磁盘的解决方案，要么不能支持本地查询，这可能不能很好地满足低延迟和高吞吐量的需求。为了解决这些问题，本文提出了一种处理大时态数据的新方法。我们的方法是Spark中基于内存的两级索引解决方案，称为ITISS。我们的解决方案的建议框架很容易理解和实施，但不会损失有效性和效率。基于所提出的框架，本文开发了有针对性的算法处理时间旅行，时间聚集，时间连接查询，分别。我们已经在Apache Spark中实现了我们的框架，扩展了Apache Spark SQL以支持声明式SQL接口，使用户能够使用几行SQL语句执行时态查询，并进行了大量的实验来验证我们解决方案的性能。基于真实的和合成数据集的实验结果一致表明，我们提出的解决方案是有效的和有竞争力的处理大时态数据。
In the real word, temporal data can be found in many applications, and it is rapidly increasing nowadays. It is urgently important and challenging to manage and operate big temporal data efficiently and effectively, due to the large volume of big temporal data and the real-time response requirement. Processing big temporal data using a distributed system is a desired choice, since a single-machine based system usually has the limited computing ability. Nevertheless, existing distributed systems or methods either are disk-based solutions, or cannot support native queries, which may not well meet the demands of low latency and high throughput. To attack these issues, this article suggests a new approach to handle big temporal data. Our approach is an In-memory based Two-level Index Solution in Spark, dubbed as ITISS. The proposed framework of our solution is easily understood and implemented, but without loss of effectiveness and efficiency. Based on the proposed framework, this article develops targeted algorithms for handling time travel, temporal aggregation, and temporal join queries, respectively. We have implemented our framework in Apache Spark, extended the Apache Spark SQL to support declarative SQL interface that enables users to perform temporal queries with a few lines of SQL statements, and conducted extensive experiments to verify the performance of our solution. The experimental results, based on both real and synthetic datasets, consistently demonstrate that our proposed solution is efficient and competitive for processing big temporal data.