Outlier detection in near-infrared spectroscopic analysis by using Monte Carlo cross-validation
Outlier detection in near-infrared spectroscopic analysis by using Monte Carlo cross-validation
复制标题
使用蒙特卡罗交叉验证进行近红外光谱分析中的异常值检测
DOI：
10.1007/s11426-008-0080-x
复制
发表时间：
2008-08-01
期刊：
影响因子：
--
通讯作者：
Shao XueGuang
中科院分区：
文献类型：
--
作者：
Liu ZhiChao;Cai WenSheng;Shao XueGuang
提出了一种用于近红外光谱分析的离群点检测方法。该方法的基本原理是，在随机检验(蒙特卡罗)交叉验证中，预测残差平方和较小的好模型或预测残差平方和较大的坏模型中出现异常值的概率应该与正常样本明显不同。该方法首先通过随机测试交叉验证建立大量的偏最小二乘模型，然后对模型进行分类，最后根据分类后模型中每个样本的累积概率识别出离群点。为了验证所提方法的有效性，对四个数据集进行了研究，其中包括三个已发表的烟叶数据集和一个大型烟叶数据集。与传统的留一法(LOO)交叉验证法相比，该方法具有较高的效率和准确性。
An outlier detection method is proposed for near-infrared spectral analysis. The underlying philosophy of the method is that, in random test (Monte Carlo) cross-validation, the probability of outliers presenting in good models with smaller prediction residual error sum of squares (PRESS) or in bad models with larger PRESS should be obviously different from normal samples. The method builds a large number of PLS models by using random test cross-validation at first, then the models are sorted by the PRESS, and at last the outliers are recognized according to the accumulative probability of each sample in the sorted models. For validation of the proposed method, four data sets, including three published data sets and a large data set of tobacco lamina, were investigated. The proposed method was proved to be highly efficient and veracious compared with the conventional leave-one-out (LOO) cross validation method.