A time-series compression technique and its application to the smart grid
A time-series compression technique and its application to the smart grid
复制标题
DOI：
10.1007/s00778-014-0368-8
复制
发表时间：
2015-04-01
期刊：
影响因子：
4.2
通讯作者：
Boehm, Klemens
中科院分区：
文献类型：
--
作者：
Eichinger, Frank;Efros, Pavel;Boehm, Klemens
在许多领域，时间序列数据的收集越来越多。一个例子是智能电力基础设施，它从智能电表等来源生成大量此类数据。虽然今天这些数据主要用于15分钟分辨率的可视化和计费，但其原始时间分辨率通常更细粒度，例如，秒这对各种分析应用程序，如短期预测，分解和可视化都很有用。然而，在许多情况下，传输和存储大量这样的细粒度数据在存储空间方面是非常昂贵的。在这篇文章中，我们提出了一个压缩技术的基础上分段回归和两种方法来描述的压缩性能。虽然我们的技术是时间序列压缩的一般方法，智能电网作为我们的运行示例和我们的评估方案。根据数据和用例场景的不同，该技术以高达5，000倍的比例压缩数据，同时保持其对分析的有用性。所提出的技术已经超过了相关的工作，并已被应用到三个现实世界的能源数据集在不同的场景。最后，我们表明，所提出的压缩技术可以实现在一个国家的最先进的数据库管理系统。
Time-series data is increasingly collected in many domains. One example is the smart electricity infrastructure, which generates huge volumes of such data from sources such as smart electricity meters. Although today these data are used for visualization and billing in mostly 15-min resolution, its original temporal resolution frequently is more fine-grained, e.g., seconds. This is useful for various analytical applications such as short-term forecasting, disaggregation and visualization. However, transmitting and storing huge amounts of such fine-grained data are prohibitively expensive in terms of storage space in many cases. In this article, we present a compression technique based on piecewise regression and two methods which describe the performance of the compression. Although our technique is a general approach for time-series compression, smart grids serve as our running example and as our evaluation scenario. Depending on the data and the use-case scenario, the technique compresses data by ratios of up to factor 5,000 while maintaining its usefulness for analytics. The proposed technique has outperformed related work and has been applied to three real-world energy datasets in different scenarios. Finally, we show that the proposed compression technique can be implemented in a state-of-the-art database management system.