Research of Association Rule Mining Parallel Processing on Very Large Multi-processors

在本研究中，我们开发了一种并行关联规则挖掘算法，并在大型多处理器(100个处理器)上实现了该算法。在第一年，我们设计了一种考虑序列数据分类的并行关联规则挖掘算法。我们对候选规则采用了哈希方法，使得我们可以很容易地在大量处理器的环境下实现可伸缩的性能。然后，利用PC集群进行了初步的实验。实验结束后，我们研究了一种考虑分类类别和频率的运行时负载均衡方法；第二年，我们用SQL实现了我们的并行关联规则挖掘算法，并在PC集群和Yendor DBMS上执行了我们的算法；然后，我们研究了算法在大型DBMS引擎上的实际使用情况。将我们的结果与用C语言编写的专用挖掘程序进行了比较，结果表明，多个结点的SQL数据挖掘算法的性能与专用程序几乎相当。最后，通过降低SQL算法与专用挖掘程序的比例，阐明了SQL并行挖掘算法的有效性。

In this research, we developed a parallel association rule mining algorithm and implemented the algorithm on the large multi-processors (100 processors). Then, we tried to make the proposed algorihtm fit for a practical use.At the first year, we designed a parallel association rule mining algorithm considering the taxonomy for sequence data. We adopt a hashing method to the candidate rules so that we can easily achieve scalable performance on the environment of large number of processors. Then, the preliminary experiment was done by using the PC cluster. After the experiment, we investigated a run time load balancing method considering the taxonomy class and the frequency.At the second year, we implemented our parallel association rule mining algorithm by SQL and executed our algorithm on the PC cluster and the yendor DBMS.Then, we investigated the pracical use of our algorithm on the large DBMS engine. Comparing our results with the special mining program written in C, we showed that the performance of SQL data mining algorithm with several nodes is almost equal to that of the special program. At last, we clarify the effectiveness of SQL parallel mining algorithm by considering reducing ratio of SQL algorithm to the special mining program.