Evaluating fidelity of lossy compression on spatiotemporal data from an IoT enabled smart farm
Evaluating fidelity of lossy compression on spatiotemporal data from an IoT enabled smart farm
复制标题
DOI：
10.1016/j.compag.2018.08.045
复制
发表时间：
2018-11-01
影响因子：
8.3
通讯作者：
Son, Seung Woo
中科院分区：
文献类型：
--
作者：
Moon, Aekyeung;Kim, Jaeyoung;Son, Seung Woo
随着智能农场应用中使用的各种物联网传感器收集的数据量不断增加，农业应用大数据的存储和处理成为一个巨大的挑战。本文的观点是，有损压缩可以释放物联网的压缩能力，因为与对应的压缩（无损压缩）相比，当正确利用物联网传感器数据的时空特性时，有损压缩可以显着减少数据量。然而，有损压缩面临着压缩过多数据从而失去数据保真度的挑战，这可能会影响数据的质量和潜在的分析结果。为了了解有损压缩对物联网数据管理和分析的影响，我们使用基于各种能量浓度的重建农业传感器数据评估了四种分类算法。具体来说，我们将三种基于转换的有损压缩机制应用于从物联网气象站以不同采样粒度收集的五个现实世界天气数据集。我们的实验结果表明，变换系数的集中能量与压缩率以及数据质量之间存在很强的正相关性。虽然我们观察到一个总体趋势，即可以以质量下降为代价实现更高的压缩比，但我们还观察到，我们评估的数据集和算法对分类准确性的影响各不相同。最后，我们表明采样粒度也会影响预测性能和压缩比方面的数据保真度。
As the volume of data collected by various IoT sensors used in smart farm applications increases, the storing and processing of big data for agricultural applications become a huge challenge. The insight of this paper is that lossy compression can unleash the power of compression to IoT because, as compared with its counterpart (a lossless one), it can significantly reduce the data volume when the spatiotemporal characteristics of IoT sensor data are properly exploited. However, lossy compression faces the challenge of compressing too much data thus losing data fidelity, which might affect the quality of the data and potential analytics outcomes. To understand the impact of lossy compression on IoT data management and analytics, we evaluated four classification algorithms with reconstructed agricultural sensor data based on various energy concentration. Specifically, we applied three transformation-based lossy compression mechanisms to five real-world weather datasets collected at different sampling granularities from IoT weather stations. Our experimental results indicate that there is a strong positive correlation between the concentrated energy of the transformed coefficients and the compression ratio as well as the data quality. While we observed a general trend where much higher compression ratios can be achieved at the cost of a decrease in quality, we also observed that the impact on the classification accuracy varies among the data sets and algorithms we evaluated. Lastly, we show that the sampling granularity also influences the data fidelity in terms of the prediction performance and compression ratio.