GeoFlink: An Efficient and Scalable Spatial Data Stream Management System
GeoFlink: An Efficient and Scalable Spatial Data Stream Management System
复制标题
DOI：
10.1109/access.2022.3154063
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
Salman Ahmed Shaikh;H. Kitagawa;Akiyoshi Matono;Komal Mariam;Kyoung-Sook Kim
中科院分区：
文献类型：
--
作者：
Salman Ahmed Shaikh;H. Kitagawa;Akiyoshi Matono;Komal Mariam;Kyoung-Sook Kim
这个时代见证了空间数据的指数增长，这是由于GPS设备的增加。空间数据如果得到及时处理，对商业企业、政府和非政府组织可能极为有用。空间数据是海量的，并且通常作为连续的数据流生成，例如，车辆跟踪数据、移动的位置数据等。为了处理如此巨大的数据流，需要高度可扩展的系统。Apache Spark Streaming、Apache Flink和Apache Samza是最先进的可扩展流处理平台之一;然而，它们缺乏空间对象、索引和查询支持。除此之外，其他可扩展的空间数据处理平台，包括GeoSpark，Spatial Hadoop不支持流式工作负载，只能处理静态或批量数据。为了填补这一空白，我们提出了GeoFlink，它扩展了Apache Flink，以支持空间对象，索引和空间数据流的连续查询。一个基于网格的索引，以支持高效的空间查询处理和有效的数据分布在分布式集群节点。GeoFlink支持对Point、LineString、Polygon、MultiPoint、MultiLineString和MultiPolygon空间对象的空间范围、空间$k$ NN和空间连接查询。此外，GeoFlink支持GeoJSON、WKT和CSV数据格式的数据流。对真实的和合成空间数据流的详细实验研究证明，GeoFlink实现了显着更高的查询吞吐量比现有的最先进的流媒体平台。
This era is witnessing an exponential growth in spatial data due to the increase in GPS-enabled devices. Spatial data can be of extreme use to commercial businesses, governments and NGOs if processed timely. Spatial data is voluminous and is usually generated as a continuous data stream, for instance, vehicles tracking data, mobile location data, etc. To process such a huge data streams, highly scalable systems are needed. Apache Spark Streaming, Apache Flink, and Apache Samza are among the state-of-the-art scalable stream processing platforms; however, they lack spatial objects, indexes, and queries support. Besides them, other scalable spatial data processing platforms including GeoSpark, Spatial Hadoop do not support streaming workloads and can only handle static or batch data. To fill this gap, we present GeoFlink which extends Apache Flink to support spatial objects, indexes and continuous queries over spatial data streams. A grid-based index is introduced to support efficient spatial query processing and effective data distribution across distributed cluster nodes. GeoFlink supports spatial range, spatial $k$ NN and spatial join queries on Point, LineString, Polygon, MultiPoint, MultiLineString, and MultiPolygon spatial objects. Besides, GeoFlink supports data streams in GeoJSON, WKT, and CSV data formats. A detailed experimental study on real and synthetic spatial data streams proves that GeoFlink achieves significantly higher query throughput than the existing state-of-the-art streaming platforms.