Efficient astronomical query processing using spark
Efficient astronomical query processing using spark
复制标题
DOI：
10.1145/3274895.3274942
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Mariem Brahem;Laurent Yeh;K. Zeitouni
中科院分区：
文献类型：
--
作者：
Mariem Brahem;Laurent Yeh;K. Zeitouni
Sky surveys represent a fundamental data source in astronomy. Today, these surveys are moving into a petascale regime produced by modern telescopes. Due to the exponential growth of astronomical data, there is a pressing need to provide efficient astronomical query processing. Our goal is to bridge the gap between existing distributed systems and high-level languages for astronomers. In this paper, we present efficient techniques for query processing of astronomical data using ASTROIDE. Our framework helps astronomers to take advantage of the richness of the astronomical data. The proposed model supports complex astronomical operators expressed using ADQL (Astronomical Data Query Language), an extension of SQL commonly used by astronomers. ASTROIDE proposes spatial indexing and partitioning techniques to better filter the data access. It also implements a query optimizer that injects spatial-aware optimization rules and strategies. Experimental evaluation based on real datasets demonstrates that the present framework is scalable and efficient.
