Distributed retrieval for massive remote sensing image metadata on spark
Distributed retrieval for massive remote sensing image metadata on spark
复制标题
DOI：
10.1109/igarss.2016.7730544
复制
发表时间：
2016-07
期刊：
影响因子：
--
通讯作者：
Fengyang Wang;Xuezhi Wang;Wenjuan Cui;Xiao Xiao-Xiao;Yuanchun Zhou;Jianhui Li
中科院分区：
文献类型：
--
作者：
Fengyang Wang;Xuezhi Wang;Wenjuan Cui;Xiao Xiao-Xiao;Yuanchun Zhou;Jianhui Li
遥感领域的海量数据不断快速增长。如何实现海量遥感影像元数据的高效存储和快速检索是一个难题。大数据技术为遥感影像元数据的存储、检索和分析提供了方便快捷的工具。根据遥感图像元数据的特点，我们首先给出了胖网格和瘦网格的新概念，并提出了一种网格索引方法Spark-Fat-Thin-Grid-Index（SFTGridIndex）。在SFTGridIndex中，索引文件和分区文件存储在HDFS中。然后我们设计了一种基于SFTGrid-Index的优化检索方法。该方法可以避免大量多边形的交集计算。本研究能够保证海量遥感影像元数据的高效存储和快速查询，并具有良好的可扩展性。
The massive data is constantly and rapidly growing in remote sensing field. How to achieve efficient storage and rapid retrieval of massive remote sensing image metadata is a difficult problem. Big Data technologies provide convenient and fast tools for the storage, retrieval and analysis of remote sensing image metadata. According to the feature of remote sensing image metadata, we first give new concepts of fat grid and thin grid and propose a grid indexing method called Spark-Fat-Thin-Grid-Index (SFTGridIndex). In SFTGridIndex, the index file and partitions files are stored in HDFS. We then design an optimized retrieval method based on SFTGrid-Index. The method can avoid the intersection computing of a large number of polygons. This research can ensure efficient storage and fast query of massive remote sensing image metadata and has good scalability.