Item Response Theory for Peer Assessment
Item Response Theory for Peer Assessment
复制标题
DOI：
10.1109/tlt.2015.2476806
复制
发表时间：
2016-04-01
影响因子：
3.7
通讯作者：
Ueno, Maomi
中科院分区：
文献类型：
--
作者：
Uto, Masaki;Ueno, Maomi
同侪评估作为一种基于建构主义理论的评估方法，近年来逐渐流行起来。然而，在同行评估中，仍然存在一个问题，即可靠性取决于评估者的特征。为此，提出了一些包含评分参数的项目反应模型。如果能准确地估计模型参数，这些模型有望提高可靠性。然而，当将其应用于实际的同行评估时，由于以下原因，参数估计的准确性会降低。(1)由于模型中包含了高维的评价参数，评价参数的数量随着评价参数数量的两倍或两倍以上而增加。2)稀疏同行评价数据参数估计的准确性很大程度上依赖于手工调优参数，称为超参数。为了解决这些问题，本文提出了一种新的同伴评估项目反应模型，该模型包含了评分参数，以保持尽可能少的评分参数。此外，本文还提出了一种基于层次贝叶斯模型的参数估计方法，该方法可以从数据中学习超参数。最后，通过仿真和实际数据实验验证了所提方法的有效性。
As an assessment method based on a constructivist approach, peer assessment has become popular in recent years. However, in peer assessment, a problem remains that reliability depends on the rater characteristics. For this reason, some item response models that incorporate rater parameters have been proposed. Those models are expected to improve the reliability if the model parameters can be estimated accurately. However, when applying them to actual peer assessment, the parameter estimation accuracy would be reduced for the following reasons. 1) The number of rater parameters increases with two or more times the number of raters because the models include higher-dimensional rater parameters. 2) The accuracy of parameter estimation from sparse peer assessment data depends strongly on hand-tuning parameters, called hyperparameters. To solve these problems, this article presents a proposal of a new item response model for peer assessment that incorporates rater parameters to maintain as few rater parameters as possible. Furthermore, this article presents a proposal of a parameter estimation method using a hierarchical Bayes model for the proposed model that can learn the hyperparameters from data. Finally, this article describes the effectiveness of the proposed method using results obtained from a simulation and actual data experiments.