A Study on Log-Data Analyzing Functions in Data-Management Systems

当今的信息系统产生了大量的各种日志数据，并且越来越需要从日志中提取有用的知识并理解目标系统中发生了什么。为了满足这些需求，本研究的重点是扩展现有的关系数据库功能，并旨在开发有效的数据库处理方法，灵活的日志数据序列的检索，日志数据转换，日志数据分析。主要成果如下：首先，我们开发了一个多维数据立方体，它支持多维分析空间中的数据挖掘。这个系统，命名为一个项目集立方体，使我们能够将一个给定的日志数据集转换成另一组有用的信息颗粒下的数据立方体模型。其次，我们开发了一个新的高效的搜索算法，命名为N-OPS，对于一个给定的数据序列，通过指定一个连续的序列模式作为一个查询。此模式是由数据库谓词p（x）组成的正则表达式，其中x是数据库记录，p（x）是关于x的SQL调用。因为N-OPS允许一般的数据库谓词，所以可以支持广泛的数据库顺序搜索任务。测试结果表明，N-OPS比传统的基于NFA的方法能够实现更少的谓词调用次数。第三，为了检测一个显着的区域在一个项目集立方体，我们开发了一种方法，应用多结构数据库操作的项目集立方体，并表明，这可以自动检测一个有趣的子序列的情况下，计算机系统日志。

Today' s information systems generate massive amounts of various log-data, and there are increasing needs of extracting useful knowledge from the logs and understanding what happen in the target systems. In order to satisfy these needs, this study is focused on extending existing relational database functions, and is aimed at developing efficient database-processing methods for flexible retrieval of log-data sequences, log-data transformation, and log-data analysis. Major results are as follows :Firstly, we developed a multi-dimensional data cube which supports data-mining in a multi-dimensional space of analysis. This system, named an Itemset cube, enables us to transform a given log-data set into another set of useful granules of information under a data cube model. Secondly, we developed a new efficient search algorithm, named N-OPS, for a given data sequence, by specifying a contiguous sequential pattern as a query. This pattern is a regular expression made of database predicates p(x), where x is a database record and p(x) is a SQL call with respect to x. Because N-OPS allows general database predicates, a wide range of database sequential search tasks can be supported. Our test showed that N-OPS can achieve much less times of predicate invocation than the traditional method based on NFA does. Thirdly, in order to detect a significant region in an itemset cube, we developed a method to apply Multi-Structural database operations to an itemset cube, and showed that this can automatically detect an interesting subsequence in case of computer system logs.