Data gathering and preprocessing apprentice

点击翻译按钮获取中文摘要

This project proposed and implemented an automatic information gathering and preprocessing system for data mining. Information gathering and preprocessing are usually a cooperative work by an information source, domain experts and mining experts. They often have to exchange and update their data. If we use an email, we will soon have a full of emails. To use a Web page we have to design an appropriate page. To overcome these difficulty, the investigators propose a new tool, WAVE (Word-of-mouth-Assisting Virtual Environment), which helps us to communicate and spread information by relaying a message like Chinese whispers. They have introduced pushing mechanism of information, a compact comment structure, a hierarchical category and visualization of information path to simplify the exchange process.To control the above preprocessing and information gathering process, they assign a weight to information delivery and a preprocessing process. In a preprocessing stage, they made an experiment to weight examples, and discovered a rule supported by many examples. For time-series data they developed a preprocessing to infer missing data and common periods, based on which a simple mining system discovers simple and easy rules.The investigators evaluated the system based on some real data sets and compare its results with ACCESS : a popular database software, and found that the proposed system processes the some case with fewer steps.