Research on Unified Discovery of Exceptions from Massive Data

点击翻译按钮获取中文摘要

The objective of this research is to study and develop a data mining method which discovers interesting exceptions from massive data in a uniform way based on various learning methods, and to justify its effectiveness by experiments with real data sets.The progress in fiscal year 2001 consists of the followings. (1) Development and refinement of various exception discovery methods including those based on support vector machines, bloomy decision tree, exception rule discovery, and boosting. We mainly worked on data squashing in order to cope with massive data. (2) Experimental evaluation of the developed exception discovery methods. We also summarized data mining contests each of which represents an occasion of systematic evaluation for various knowledge discovery methods with a set of common problems. (3) Planning and investigation of a unified exception discovery method. We also performed a novel type of worst-case analysis of rule discovery as a foundation of automated discovery.In fiscal year 2002, we first developed a unified exception rule discovery and implemented it on computers. Important issues in the integration include usefulness of discovered knowledge and effectiveness of the approach based on the experimental results in the previous fiscal year, In the development, each exception discovery method was refined if necessary. According to the results of preliminary experiments, we have chosen the unified exception discovery method which employs exception rule discovery method and outlier detection method based on boosting as our final system among the exception discovery methods developed and refined in the last fiscal year. In the latter half of this fiscal year, we performed final experiments in which we applied the implemented unified exception discovery method to preprocessed massive data.