Research of Association Rule Mining Parallel Processing on Very Large Multi-processors

超大型多处理器上关联规则挖掘并行处理研究

• 批准号: 11558030
• 负责人: KITSUREGAWA Masaru
• 金额: $ 8.7万
• 依托单位: University of Tokyo
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (B).
• 财政年份: 1999
• 资助国家: 日本
• 起止时间: 1999 至 2000
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-11558030/
• 关键词: data mining; parallel database processing; distributed processing; 並列データベース

In this research, we developed a parallel association rule mining algorithm and implemented the algorithm on the large multi-processors (100 processors). Then, we tried to make the proposed algorihtm fit for a practical use.At the first year, we designed a parallel association rule mining algorithm considering the taxonomy for sequence data. We adopt a hashing method to the candidate rules so that we can easily achieve scalable performance on the environment of large number of processors. Then, the preliminary experiment was done by using the PC cluster. After the experiment, we investigated a run time load balancing method considering the taxonomy class and the frequency.At the second year, we implemented our parallel association rule mining algorithm by SQL and executed our algorithm on the PC cluster and the yendor DBMS.Then, we investigated the pracical use of our algorithm on the large DBMS engine. Comparing our results with the special mining program written in C, we showed that the performance of SQL data mining algorithm with several nodes is almost equal to that of the special program. At last, we clarify the effectiveness of SQL parallel mining algorithm by considering reducing ratio of SQL algorithm to the special mining program.

在本研究中，我们提出一个平行关联规则探勘演算法，并在大型多处理器（100处理器）上实作。第一年，我们设计了一个考虑分类的序列数据并行关联规则挖掘算法。对候选规则采用了哈希方法，使得在处理器数量较多的环境下，可以很容易地获得可扩展的性能。然后，利用PC集群进行了初步的实验。第二年，我们用SQL语言实现了我们的并行关联规则挖掘算法，并在PC机群和Yendor数据库管理系统上运行了我们的算法，然后在大型数据库引擎上进行了实际应用的研究。通过与用C语言编写的专用挖掘程序的比较，表明多节点SQL数据挖掘算法的性能与专用挖掘程序的性能相当。最后，从SQL算法对特定挖掘程序的缩减率来说明SQL并行挖掘算法的有效性。

项目成果

Takahiko Shintani,Masaru Kitsuregawa: "Parallel Generalized Association rule Mining on Large Scale PC Cluster"Proceedings of Workshop on Large-Scale Parallel KDD Systems. 35-44 (1999)

Takahiko Shintani、Masaru Kitsurekawa：“大规模 PC 集群上的并行广义关联规则挖掘”大规模并行 KDD 系统研讨会论文集。

Takahiko Shintani and Masaru Kitsuregawa: "Parallel Generalized Association Rule Mining on Large Scale PC Cluster"Large-Scale Parallel Data Mining ISBN 3-540-67194-3. 145-160 (2000)

Takahiko Shintani 和 Masaru Kitsurekawa：“大规模 PC 集群上的并行广义关联规则挖掘”大规模并行数据挖掘 ISBN 3-540-67194-3。

Masaru Kitsuregawa, Takahiko Shintani, Takeshi Yoshizawa, Iko Pramudiono: "Web Log Mining and Parallel SQL Based Execution (Key note address)"International Workshop on Databases in Networked Information Systems (DNIS2000),, University of Aizu. 20-32 (2000

Masaru Kitsurekawa、Takahiko Shintani、Takeshi Yoshizawa、Iko Pramudiono：“Web 日志挖掘和基于 SQL 的并行执行（主题演讲）”网络信息系统数据库国际研讨会 (DNIS2000)，会津大学。

Iko Pramdiono, Takahiko Shintani, Takayuki Tamura, Masaru Kitsuregawa: "Parallel SQL Based Association Rule Mining on Large Scale PC Cluster : Performance Comparison with Directly Coded C Implementation"Proceedings of Third Pacific-Asia Conference on Know

Iko Pramdiono、Takahiko Shintani、Takayuki Tamura、Masaru Kitsurekawa：“大规模 PC 集群上基于并行 SQL 的关联规则挖掘：与直接编码 C 实现的性能比较”第三届亚太亚洲 Know 会议论文集

Masaru Kitsuregawa, Takahiko Shintani, Takeshi Yoshizawa, Iko Pramudiono: "Web Log Mining and Parallel SQL Based Execution (Key note address)"International Workshop on Databases in Networked Information Systems (DNIS2000). 20-32 (2000)

Masaru Kitsurekawa、Takahiko Shintani、Takeshi Yoshizawa、Iko Pramudiono：“Web 日志挖掘和基于 SQL 的并行执行（主题演讲）”网络信息系统数据库国际研讨会 (DNIS2000)。