Rule extraction by a structural learning of neural networks

点击翻译按钮获取中文摘要

In extracting rules from continuous valued inputs, the balance between mean square output error and the complexity of rules is important. Information criteria such as AIC represents this trade-off. In a structural learning with forgetting (SLF), the amount of forgetting is determined by minimizing AIC.However, SLF alone cannot produce rules of appropriate complexity. To overcome this difficulty, neural networks of various degrees of, complexity are trained. The degree of complexity, here, is defined by the maximum number of incoming connections to each hidden unit. From among these, the one with the smallest AIC is selected as optimal. Since outputs of hidden units are binary owing to the learning with hidden units clarification, incoming connection weights to each hidden unit determine the corresponding discriminating hyperplane. A logical combination of these hyperplanes provides rules. Furthermore, comparison with C4.5 popular in machine learning. Also comparison is made with KT met … More hod proposed by Fu. C4.5 and KT method can only produce rules with only one attribute at each term. On the other hand, the proposed method can produce rules of various complexities.The first task is to divide a two-dimensional plane into two categories. In this case, rules with only one attribute at each term is not a natural representation. C4.5 generates many simple rules, but the proposed method can explain all data by 6 rules with two attributes. The second task is the classification of irises into 3 categories : setosa, versicolor, and virginica. Three rules with at most 3 attributes can explain 148 samples out of 150. The third task is the diagnosis of thyroid functioning into 3 classes : normal, hypo and hyper functioning. In this case 4 rules with at most 2 attributes can explain all 215 samples. Furthermore, the number of classification errors is smaller than those by C4.5 and KT method.Concerning rule extraction from both continuous and discrete inputs and that from continuous inputs and outputs, satisfactory results are not yet obtained due to inherent difficlty. These are left for further study. Less