TrioStat: Online Workload Estimation in Distributed Spatial Data Streaming Systems
TrioStat: Online Workload Estimation in Distributed Spatial Data Streaming Systems
复制标题
DOI：
10.1145/3397536.3422220
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Anas Daghistani;W. Aref;A. Ghafoor
中科院分区：
文献类型：
--
作者：
Anas Daghistani;W. Aref;A. Ghafoor
支持GPS的设备和物联网（IoT）的广泛传播增加了每秒生成的空间数据的量。当前的空间数据尺度无法使用集中式系统来处理。这导致了分布式空间数据流系统的开发，该系统可以在实时进行大量流空间数据进行扩展。分布式流媒体系统的性能依赖于即使是如何在其机器之间分发工作负载。但是，估计每台机器的工作量是一项挑战，因为空间数据和查询流偏斜并随着时间和用户的兴趣而迅速变化。此外，分布式空间流系统通常无法维持全局系统工作负载状态，因为它需要高网络和处理开销，以便从系统中的计算机收集。本文介绍了triostat；一种在线工作负载估计技术依赖于概率模型来估计分布式空间数据流系统中分区和机器的工作量。与集中式单元收集和交换统计数据是不可行的，因为它需要高网络开销。取而代之的是，TrioStat使用分散的技术在每台计算机中的本地实时收集和维护所需的统计信息。 TrioStat使分布式的空间数据流系统比较机器的工作负载以及数据分区的工作负载。 Triostat需要最小的网络和存储开销。此外，所需的存储在系统的机器上分布。
The wide spread of GPS-enabled devices and the Internet of Things (IoT) has increased the amount of spatial data being generated every second. The current scale of spatial data cannot be handled using centralized systems. This has led to the development of distributed spatial data streaming systems that scale to process in real-time large amounts of streamed spatial data. The performance of distributed streaming systems relies on how even the workload is distributed among their machines. However, it is challenging to estimate the workload of each machine because spatial data and query streams are skewed and rapidly change with time and users' interests. Moreover, a distributed spatial streaming system often does not maintain a global system workload state because it requires high network and processing overheads to be collected from the machines in the system. This paper introduces TrioStat; an online workload estimation technique that relies on a probabilistic model for estimating the workload of partitions and machines in a distributed spatial data streaming system. It is infeasible to collect and exchange statistics with a centralized unit because it requires high network overhead. Instead, TrioStat uses a decentralised technique to collect and maintain the required statistics in real-time locally in each machine. TrioStat enables distributed spatial data streaming systems to compare the workloads of machines as well as the workloads of data partitions. TrioStat requires minimal network and storage overhead. Moreover, the required storage is distributed across the system's machines.