Interrater agreement reconsidered:: An alternative to the rwg indices
Interrater agreement reconsidered:: An alternative to the rwg indices
复制标题
DOI：
10.1177/1094428105275376
复制
发表时间：
2005-04-01
影响因子：
9.5
通讯作者：
Hauenstein, NMA
中科院分区：
文献类型：
--
作者：
Brown, RD;Hauenstein, NMA
对于连续结构，最常用的评价者间一致性指数（r（wg（1）））可能存在问题。通常，r（wg（1））是在假设均匀分布代表不一致的情况下估计的。作者回顾了这种统一的零r（wg（1））指数的局限性，并讨论了测量评价者间一致性的替代方法。提出了一种新的评价者间一致性统计量a（wg（1））。作者推导出a（wg（1））统计量，并证明a（wg（1））类似于科恩的kappa，这是名义数据的评分者间一致性指数。比较了基于一致r（wg（1））和a（wg（1））的一致性估计，并讨论了最小样本容量和实际显著性水平等问题。作者最后给出了关于假设一致空值时使用r（wg（1））/r（wg（J））、不假设一致空值的指数、a（wg（1））/a（wg（J））指数以及评价者间一致性的普遍性估计的建议。
For continuous constructs, the most frequently used index of interrater agreement (r(wg(1))) can be problematic. Typically, r(wg(1)) is estimated with the assumption that a uniform distribution represents no agreement. The authors review the limitations of this uniform null r(wg(1)) index and discuss alternative methods for measuring interrater agreement. A new interrater agreement statistic, a(wg(1)), is proposed. The authors derive the a(wg(1)) statistic and demonstrate that a(wg(1)) is an analogue to Cohen's kappa, an interrater agreement index for nominal data. A comparison is made between agreement estimates based on the uniform r(wg(1)) and a(wg(1)), and issues such as minimum sample size andpractical significance levels are discussed. The authors close with recommendations regarding the use of r(wg(1))/r(wg(J)) when a uniform null is assumed, indices that do not assume a uniform null, a(wg(1))/a(wg(J)) indices, and generalizability estimates of interrater agreement.