Using pooled kappa to summarize interrater agreement across many items
Using pooled kappa to summarize interrater agreement across many items
复制标题
DOI：
10.1177/1525822x08317166
复制
发表时间：
2008-08-01
期刊：
影响因子：
1.7
通讯作者：
Teleki, Stephanie S.
中科院分区：
文献类型：
--
作者：
De Vries, Han;Elliott, Marc N.;Teleki, Stephanie S.
我们提出了kappa的合并估计量，这是在总结具有许多项目但很少受试者的定性数据的评价者间一致性时的一个有效估计量。我们通过模拟建议和替代（平均Kappa）估计值来评估此估计值，随后应用我们的方法计算汇总和平均Kappa超过2,176个评分项目，来自CAHPS赞助商的六个半结构化访谈。建议合并kappa估计有效地总结了评价者之间的协议域。它的适用范围更广，比简单地平均项目级kappas更好地利用了稀缺的科目。
We propose the pooled estimator of kappa, an efficient estimator when summarizing the interrater agreement for qualitative data with many items but few subjects. We evaluate this estimator through a simulation of proposed and alternative (average kappa) estimators and subsequently apply our method to calculate pooled and average kappas over 2,176 rated items from six semistructured interviews with sponsors of the CAHPS. The proposed pooled kappa estimator efficiently summarizes interrater agreement by domain. It is more widely applicable and makes better use of scarce subjects than simply averaging item-level kappas.