Efficient Transmission and Reconstruction of Dependent Data Streams via Edge Sampling
Efficient Transmission and Reconstruction of Dependent Data Streams via Edge Sampling
复制标题
DOI：
10.1109/ic2e55432.2022.00013
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Joel Wolfrath;A. Chandra
中科院分区：
文献类型：
--
作者：
Joel Wolfrath;A. Chandra
由于智能设备的普及和对实时分析的需求，数据流处理是一个越来越重要的话题。地理分布的流传输系统（其中基于云的查询利用来自多个分布式设备的数据流）面临挑战，因为广域网（WAN）带宽通常稀缺或昂贵。边缘计算允许我们通过利用靠近设备的资源来解决这些带宽成本，例如对传入的数据流进行采样，这会降低下游查询的准确性以降低整体传输成本。在本文中，我们利用了数据流之间的相关性可能存在于位于同一地理区域的设备之间的事实。利用这一洞察力，我们开发了一个混合边缘云系统，该系统在边缘采样和云中缺失值的估计之间进行系统性权衡，以减少WAN上的流量。我们提出了一个优化框架，它计算边缘的样本大小，并系统地限制我们可以在云中估计的样本数量，因为流之间的相关性很强。我们对三个真实数据集的评估表明，与现有的采样技术相比，我们的系统可以在多个聚合查询上提供相当的错误率，同时减少27- 42%的WAN流量。
Data stream processing is an increasingly important topic due to the prevalence of smart devices and the demand for real-time analytics. Geo-distributed streaming systems, where cloud-based queries utilize data streams from multiple distributed devices, face challenges since wide-area network (WAN) bandwidth is often scarce or expensive. Edge computing allows us to address these bandwidth costs by utilizing resources close to the devices, e.g. to perform sampling over the incoming data streams, which trades downstream query accuracy to reduce the overall transmission cost. In this paper, we leverage the fact that correlations between data streams may exist across devices located in the same geographical region. Using this insight, we develop a hybrid edge-cloud system which systematically trades off between sampling at the edge and estimation of missing values in the cloud to reduce traffic over the WAN. We present an optimization framework which computes sample sizes at the edge and systematically bounds the number of samples we can estimate in the cloud given the strength of the correlation between streams. Our evaluation with three real-world datasets shows that compared to existing sampling techniques, our system could provide comparable error rates over multiple aggregate queries while reducing WAN traffic by 27-42%.