階層型ニューラルネットによる非線形多変量データ解析に関する研究

在将前馈神经网络模型应用于回归和分类问题时，引入了网络输出的概率解释，构造了模型的似然原理。首先，我们提出了一个单二进制输出的前馈神经网络模型，该模型可以看作是有序逻辑回归模型的扩展。利用二元响应的arcsin变换，给出了网络模型的似然函数和学习算法。证明了基于arcsin变换的似然函数最大化可以使神经网络模型中的平方和误差函数最小化。通过残差指数图与有序逻辑回归模型进行比较，对该方法进行了评价。非线性判别分析的分类问题也研究了使用前馈神经网络的单一或多个输出。本文推导了平方和误差函数和Kullback-Leibler测度两类误差函数之间的关系定理。为了优化复杂的网络模型，提出了基于似然方法的统计推断。基于噪声正态分布假设的平方和AIC，被用于从多个竞争模型中选择最佳模型。我们提出了一种基于自举法的替代信息准则，用于确定适当数量的隐藏单元。我们还给出了实际错误率的自举估计，因为当训练样本相对于参数数量较小时，过量误差估计很重要。结果表明，该方法比Fisher判别分析和CART （Classification and Regression Trees）方法的错误率更小。由于噪声的学习，复杂网络模型并不总能达到良好的泛化效果。因此，我们开发了使用似然比统计量的连接权值修剪算法，以检验连接权值的显著性。我们还使用似然比统计来选择预测变量的最佳子集。最后，从降维的角度研究了判别分析与用于分类的前馈神经网络之间的关系。通过类比多维典型判别分析中的典型分数技术，导出了压缩分数作为预测变量的线性组合。由于数据的图形化表示，它允许我们减少信息的维度。少

In application of feed-forward neural network models to regression and classification problems, we introduce the probabilistic interpretations of network outputs and construct the likelihood principle of the models. We first, present a feed-forward neural network model for a single binary output, which can be regarded as an extension of ordinal logistic regression models. With the arc-sine transformation for binary response, we provide the likelihood function of network models and the learning algorithm. It is proved that maximizing the likelihood function based on the arc-sine transformation results in minimizing the sum-of-squares error function in the neural network model. The proposed method is evaluated by comparison with ordinal logistic regression model through the index plots of the residuals.Non-linear discriminant analysis in classification problems is also investigated by using feed-forward neural networks with a single or multiple outputs. We derive the theorem of the relat … More ionship of two types of error function, i. e., the sum-of-squares error function and the Kullback-Leibler measure. Statistical inference based on the likelihood approach is then formulated in order to optimize a complex network model. AIC with the sum-of-squares, which is based on the assumption that the noise is normally distributed, has been used for selection of a best model among several competing models. We suggest an alternative information criterion based on the bootstrap method for determination of the appropriate number of hidden units. We also present the bootstrap estimates of the actual error rates because excess error estimation is important when the training sample is small relative to the number of parameters. It is shown that the proposed method has smaller error rates than those by Fisher's discriminant analysis and CART (Classification And Regression Trees).Complex network models do not always achieve good generalization due to the learning of the noise. We thus develop the pruning algorithm of the connection weights using the likelihood-ratio statistic in order to test the significance of the connection weights. We also use the likelihood-ratio statistic for selecting the best subset of predictor variables. From the point of dimensionality reduction, we finally study the relationship between discriminant analysis and fee-forward neural networks used for classification. By analogy with the technique for the canonical scores in the multi-dimensional canonical discriminant analysis, we derive the compression scores as the linear combination of predictor variables. It allows us to reduce the dimensions of the information due to graphical presentation of the data. Less