Classifier performance estimation under the constraint of a finite sample size: resampling schemes applied to neural network classifiers.
Classifier performance estimation under the constraint of a finite sample size: resampling schemes applied to neural network classifiers.
复制标题
有限样本大小约束下的分类器性能估计：应用于神经网络分类器的重采样方案。
DOI：
10.1016/j.neunet.2007.12.012
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
Hadjiiski,Lubomir
中科院分区：
文献类型：
--
作者：
Sahiner,Berkman;Chan,Heang-Ping;Hadjiiski,Lubomir
In a practical classifier design problem the sample size is limited, and the available finite sample needs to be used both to design a classifier and to predict the classifier’s performance for the true population. Since a larger sample is more representative of the population, it is advantageous to design the classifier with all the available cases, and to use a resampling technique for performance prediction. We conducted a Monte Carlo simulation study to compare the ability of different resampling techniques in predicting the performance of a neural network (NN) classifier designed with the available sample. We used the area under the receiver operating characteristic curve as the performance index for the NN classifier. We investigated resampling techniques based on the cross-validation, the leave-one-out method, and three different types of bootstrapping, namely, the ordinary, .632, and .632+ bootstrap. Our results indicated that, under the study conditions, there can be a large difference in the accuracy of the prediction obtained from different resampling methods, especially when the feature space dimensionality is relatively large and the sample size is small. Although this investigation is performed under some specific conditions, it reveals important trends for the problem of classifier performance prediction under the constraint of a limited data set.
