Study of High-speed Data Mining Algorithms from Massive Data Streams

在本研究中，我们研究了从海量半结构化数据流中提取有用信息的高速在线知识发现系统。特别是在这一年里，作为理论研究，我们进一步扩展了在线流的高效模式匹配和模式发现方法的理论。作为应用研究，我们在一个庞大的组织中对实际高速网络的网络数据进行了一系列的采集和分析实验。我们也公布了过去三年的研究成果。具体来说，我们在以下几个问题上进行了研究：(1)半结构化数据综述：我们在一家学术期刊上总结并发表了一篇关于流数据挖掘的综述，这是本项目近三年来一直在研究的。(2)半结构化数据的流模式匹配技术研究：我们提出了一种对位并行技术进行水平通配符的树模式匹配的高效方法，这将极大地提高XPath和XQuery模式匹配语言在处理海量XML数据时的速度。(3)半结构化数据的顺序和流模式发现技术研究：针对各类复杂模式/主题，我们开发了高效的算法从海量数据流中发现感兴趣的模式。在这一年，我们还发表了去年开发的模式发现算法。从真实海量网络数据中进行知识发现的实证研究：作为应用，我们在九州大学进行了一系列针对中型组织的高速大规模网络数据采集和在线分析的调查。这些实验将为未来高速流数据的高效模式匹配/发现算法的研究提供启示。

In this research, we investigated high-speed online knowledge discovery system for extracting useful information from massive semi-structured data streams. Particularly in this year, as theoretical researches, we extended further the theory of efficient pattern matching and pattern discovery methods for online streams. As application studies, we made a series of experiments on collection and analysis of network data from real high-speed networks in a huge organization. We have also published the results obtained in the research period of the last three years. In particular, we proceed the studies on the following issues:(1)Survey on semi-structured data : We have summarized and published a survey on stream data mining in an academic journal, which has been studied through this project for the last three years.(2)Study on streaming pattern matching technology for semi-structured data : We developed an efficient method for performing tree pattern matching with horizontal wildcards by bit parallel technology, which potentially gives drastic speed-up for Xpath and XQuery pattern matching languages for huge XML data.(3)Study on sequential and streaming pattern discovery technology for semi-structured data : We developed efficient algorithms for finding interesting patterns from massive data streams for various classes of complex patterns/motifs. In this year, we also published pattern discovery algorithms developed in the last year. Also, one of them got awarded for 2004 JSAI SIG AWARD.(4)Empirical study on knowledge discovery from real massive network data : As applications, we performed a series of surveys on data collection and online analysis of high-speed large-scale network for middle sized organization at Kyushu University. These experiments will give insights for future research on the development of efficient pattern matching/discovery algorithms for high-speed streaming data.