Research on model selection of multi-layer perceptron

在这项研究中，我们专注于在多层感知器的基函数的可变性，并建立了一个模型选择方法的情况下，基函数是从一个有限的功能集，其中的候选人的数量等于数据的数量。在所提出的方法中，在集合中的函数正交化，并估计其系数。通过阈值化将无效系数设置为零。然后逆变换得到原始基函数的权值。该方法中的阈值水平是基于基函数的可变性而推导的，因此在理论上是合理的。我们可以根据正交化的方法获得平滑的输出和/或稀疏表示。该方法适用于在一定训练算法下的多层感知器。

In this research, we focus on a variability of basis functions in multi-layer perceptron and establish a model selection method for the case where basis functions are selected from a finite set of functions, in which the number of candidates is equal to the number of data. In the proposed method, functions in the set are orthogonalized and their coefficients are estimated. Inefficient coefficients are set to zero by thresholding. Then inverse transform yields the weights of original basis functions. The threshold level in this method is theoretically reasonable since it derived based on the variability of basis functions. We can obtain a smooth output and/or sparse representation depending on the method of orthogonalization. The method is applicable to multi-layer perceptron under a certain training algorithm.