Testing the Difference of Correlated Agreement Coefficients for Statistical Significance
Testing the Difference of Correlated Agreement Coefficients for Statistical Significance
复制标题
DOI：
10.1177/0013164415596420
复制
发表时间：
2016-08-01
影响因子：
2.7
通讯作者：
Gwet, Kilem L.
中科院分区：
文献类型：
--
作者：
Gwet, Kilem L.
本文讨论了两个相关的一致性系数之间的差异的统计显著性测试的问题。许多作者提出了测试两个相关的Kappa系数之间的差异的方法，这需要使用reservation方法或使用先进的统计建模技术。在这篇文章中，我们提出了一种技术类似于经典的成对t检验的手段，这是基于一个大样本的线性近似的协议系数。我们用几个已知的一致系数来说明这种技术的使用，包括Cohen的kappa，Gwet的AC（1），Fleiss的广义kappa，Conger的广义kappa，Krippendorff的alpha和Brenann-Prediger系数。所提出的方法是非常灵活的，可以容纳几种类型的系数之间的相关结构，既不需要先进的统计建模技能，也不需要相当多的计算机编程经验。通过蒙特卡罗仿真验证了该方法的有效性。
This article addresses the problem of testing the difference between two correlated agreement coefficients for statistical significance. A number of authors have proposed methods for testing the difference between two correlated kappa coefficients, which require either the use of resampling methods or the use of advanced statistical modeling techniques. In this article, we propose a technique similar to the classical pairwise t test for means, which is based on a large-sample linear approximation of the agreement coefficient. We illustrate the use of this technique with several known agreement coefficients including Cohen's kappa, Gwet's AC(1), Fleiss's generalized kappa, Conger's generalized kappa, Krippendorff's alpha, and the Brenann-Prediger coefficient. The proposed method is very flexible, can accommodate several types of correlation structures between coefficients, and requires neither advanced statistical modeling skills nor considerable computer programming experience. The validity of this method is tested with a Monte Carlo simulation.