Research on model selection of multi-layer perceptron

点击翻译按钮获取中文摘要

In this research, we focus on a variability of basis functions in multi-layer perceptron and establish a model selection method for the case where basis functions are selected from a finite set of functions, in which the number of candidates is equal to the number of data. In the proposed method, functions in the set are orthogonalized and their coefficients are estimated. Inefficient coefficients are set to zero by thresholding. Then inverse transform yields the weights of original basis functions. The threshold level in this method is theoretically reasonable since it derived based on the variability of basis functions. We can obtain a smooth output and/or sparse representation depending on the method of orthogonalization. The method is applicable to multi-layer perceptron under a certain training algorithm.