MEMORY MANAGEMENT FOR BIG DATA MINING -- CACHE HIT RATE ESTIMATION OF LESSFU
MEMORY MANAGEMENT FOR BIG DATA MINING -- CACHE HIT RATE ESTIMATION OF LESSFU
复制标题
大数据挖掘的内存管理——LESSFU的缓存命中率估计
DOI：
10.1016/j.protcy.2014.10.214
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
K. Yoshida
中科院分区：
文献类型：
--
作者：
Hirotsugu Yamamoto;Yuka Tomiyama;and Shiro Suyama;K. Yoshida
我们开发了一个网络监控器，它可以从Internet主干网流量中发现Internet病毒发送的IP数据包。监控器的主要组成部分是一个每秒处理10M事务的数据挖掘引擎。虽然理论上数据挖掘引擎需要分析超过200G字节的数据，但是一种名为LessFU的内存管理策略删除了不必要的数据以实现高效的处理。我们过去的实验，使用真实的互联网流量显示了我们的方法的优势。然而，LessFU的该高速缓存命中率目前还没有一种评估方法。针对该高速缓存命中率对数据挖掘结果造成严重后果的问题，提出了一种估计LessFU的该高速缓存命中率的方法。实验结果表明，该方法的优点，本文也报道。
We have developed a network monitor which can find IP packets sent by Internet Virus from Internet backbone traffic. A data mining engine which can handle 10 M transactions per second is the main component of the monitor. Although the data mining engine have to analyze over 200G byte data in theory, a memory management strategy named LessFU removes non-essential data to realize efficient processing. Our past experiments which use real Internet traffic shows the advantage of our approach. However, there exits no method to evaluate the cache hit rate of LessFU. Since the cache hit rate results in serious consequences on the data mining results, this paper proposes a method to estimate the cache hit rate of LessFU. The experimental results which show the advantage of the proposed method are also reported in this paper.