A Nonparametric Approach to Estimate Classification Accuracy and Consistency
A Nonparametric Approach to Estimate Classification Accuracy and Consistency
复制标题
DOI：
10.1111/jedm.12048
复制
发表时间：
2014-09-01
影响因子：
1.3
通讯作者：
Cheng, Ying
中科院分区：
文献类型：
--
作者：
Lathrop, Quinn N.;Cheng, Ying
当分类的分数出现在总分量表上时，用于估计分类准确性（CA）和分类一致性（CC）的流行方法需要关于测试分数的参数形式或关于参数反应模型（例如项目反应理论（IRT））的假设。本文开发了一种非参数估计CA和CC的方法，通过用Ramsay核平滑项目响应函数的修改版本替换Lee分类指数中参数IRT模型的作用。在模拟研究中，采用不同的生成IRT模型、测试长度和能力分布，对非参数CA和CC指数的性能进行了测试。CA的非参数方法往往优于Lee的方法和利文斯顿和刘易斯的方法，显示出对非正态性的鲁棒性的模拟能力。当能力分布为非正态分布时，非参数CC指数的性能与Lee方法相似，优于利文斯顿和刘易斯方法。
When cut scores for classifications occur on the total score scale, popular methods for estimating classification accuracy (CA) and classification consistency (CC) require assumptions about a parametric form of the test scores or about a parametric response model, such as item response theory (IRT). This article develops an approach to estimate CA and CC nonparametrically by replacing the role of the parametric IRT model in Lee's classification indices with a modified version of Ramsay's kernel-smoothed item response functions. The performance of the nonparametric CA and CC indices are tested in simulation studies in various conditions with different generating IRT models, test lengths, and ability distributions. The nonparametric approach to CA often outperforms Lee's method and Livingston and Lewis's method, showing robustness to nonnormality in the simulated ability. The nonparametric CC index performs similarly to Lee's method and outperforms Livingston and Lewis's method when the ability distributions are nonnormal.