Fill-in the gaps: Spatial-temporal models for missing data
Fill-in the gaps: Spatial-temporal models for missing data
复制标题
DOI：
10.23919/cnsm.2017.8255983
复制
发表时间：
2017-11
期刊：
影响因子：
--
通讯作者：
Ji Xue;Bin Nie;E. Smirni
中科院分区：
文献类型：
--
作者：
Ji Xue;Bin Nie;E. Smirni
有效的工作负载特征描述和预测对于高效且主动地管理大型系统至关重要。系统管理主要依赖于底层系统跟踪机制所提供的工作负载信息，这些机制将与系统相关的事件记录在日志文件中。然而，由于各种原因，此类跟踪机制可能会暂时失效，从而在数据跟踪中产生“漏洞”。这种数据缺失现象严重阻碍了数据分析的有效性。在本文中，我们研究了从一家服务提供商的数据中心的6000个物理机盒上托管的8万多个虚拟机（VM）收集到的真实世界数据跟踪情况。我们发现，位于同一物理机盒上的虚拟机的使用序列彼此之间呈现出很强的相关性，并且大多数虚拟机使用序列呈现出时间模式。通过利用所观察到的空间和时间依赖性，我们提出了一种数据填充方法来预测虚拟机使用序列中的缺失数据。使用实际跟踪数据进行的详细评估表明，所提出的方法足够准确，因为它实现了平均20%的绝对百分比误差。我们还通过一个用例说明了它的实用性。
Effective workload characterization and prediction are instrumental for efficiently and proactively managing large systems. System management primarily relies on the workload information provided by underlying system tracing mechanisms that record system-related events in log files. However, such tracing mechanisms may temporarily fail due to various reasons, yielding “holes” in data traces. This missing data phenomenon significantly impedes the effectiveness of data analysis. In this paper, we study real-world data traces collected from over 80K virtual machines (VMs) hosted on 6K physical boxes in the data centers of a service provider. We discover that the usage series of VMs co-located on the same physical box exhibit strong correlation with one another, and that most VM usage series show temporal patterns. By taking advantage of the observed spatial and temporal dependencies, we propose a data-filling method to predict the missing data in the VM usage series. Detailed evaluation using trace data in the wild shows that the proposed method is sufficiently accurate as it achieves an average of 20% absolute percentage errors. We also illustrate its usefulness via a use case.