A new standard for the analysis and design of replication studies
A new standard for the analysis and design of replication studies
复制标题
DOI：
10.1111/rssa.12493
复制
发表时间：
2019-12-26
影响因子：
2
通讯作者：
Held, Leonhard
中科院分区：
文献类型：
--
作者：
Held, Leonhard
提出了一个新的标准，用于重复研究的证据评估。该方法结合了一个特定的反向贝叶斯技术与先验预测尾部概率来定义复制成功。该方法产生了一个复制成功的定量测量，称为怀疑p值。怀疑p值整合了原始和复制研究的传统显著性，并比较了各自的效应量。它合并了原始和复制效应估计值的不确定性，如果忽略原始效应估计值的不确定性，则可简化为复制研究的普通p值。建议的框架也可以用来确定的权力或所需的复制样本大小，以实现复制成功。数值计算强调了如果原始研究的证据仅是提示性的，则实现复制成功的难度。从开放科学合作项目的心理科学的可复制性数据的应用程序说明了所提出的方法。
A new standard is proposed for the evidential assessment of replication studies. The approach combines a specific reverse Bayes technique with prior-predictive tail probabilities to define replication success. The method gives rise to a quantitative measure for replication success, called the sceptical p-value. The sceptical p-value integrates traditional significance of both the original and the replication study with a comparison of the respective effect sizes. It incorporates the uncertainty of both the original and the replication effect estimates and reduces to the ordinary p-value of the replication study if the uncertainty of the original effect estimate is ignored. The framework proposed can also be used to determine the power or the required replication sample size to achieve replication success. Numerical calculations highlight the difficulty of achieving replication success if the evidence from the original study is only suggestive. An application to data from the Open Science Collaboration project on the replicability of psychological science illustrates the methodology proposed.