Methods for assessing replication

重复先前的发现和结果是科学的一个基本特征，也是支持科学可以自我纠正这一主张的逻辑的一部分。然而，关于研究复制的方法的研究很少。涉及荟萃分析和系统综述的研究更常见，这些研究总结了一系列研究。然而，一系列实验研究的发现是否可以相互复制的问题却很少受到关注。对于成功的重复性研究，没有明确定义和广泛接受的定义，也没有统计文献为如何设计单次重复性研究或一组重复性研究提供方法学指导。这里提出的研究建立了这种急需的方法论。该项目由探索研究PreK-12计划资助，该计划为STEM创新和方法的研究和开发提供资金。本研究的目标是形式化关于复制这一重要概念的主观观念，为评估复制研究提供统计分析，为评估复制研究的结论性提供属性，并为设计结论性和高效率的复制研究方案提供原则。它解决了三个基本问题。首先是如何定义重复性：确切地说，一系列研究的结果相互复制意味着什么？其次，给定复制的定义，应该做哪些统计分析来决定研究的集合是否相互复制，这些分析的属性是什么（例如，敏感性或统计能力）？第三，如何设计一个或多个重复性研究来为重复性问题提供结论性的答案？该项目通过提供统计工具来评估实验结果的可重复性，评估复制尝试的结论性，以及开发软件来帮助规划复制研究程序，从而为科学发现的可重复性提供确凿的证据，从而对一系列实证科学产生潜在的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Replication of prior findings and results is a fundamental feature of science and is part of the logic supporting the claim that science is self-correcting. However, there is little prior research on the methodology for studying replication. Research involving meta-analysis and systematic reviews that summarizes a collection of research studies is more common. However, the question of whether the findings from a set of experimental studies replicate one another has received less attention. There is no clearly defined and widely accepted definition of a successful replication study or statistical literature providing methodological guidelines on how to design single replication studies or a set of replication studies. The research proposed here builds this much needed methodology. This project is funded by the Discovery Research PreK-12 Program, which funds research and development on STEM innovations and approaches.The goal of this project is to formalize subjective ideas about the important concept of replication, provide statistical analyses for evaluating replication studies, provide properties for evaluating the conclusiveness of replication studies, and provide principles for designing conclusive and efficient programs of replication studies. It addresses three fundamental problems. The first is how to define replication: What, precisely, should it mean to say that the results in a collection of studies replicate one another? Second, given a definition of replication, what statistical analyses should be done to decide whether the collection of studies replicate one another and what are the properties of these analyses (e.g., sensitivity or statistical power)? Third, how should one or more replication studies be designed to provide conclusive answers to questions of replication? The project has the potential for impact on a range of empirical sciences by providing statistical tools to evaluate the replicability of experimental findings, assessing the conclusiveness of replication attempts, and developing software to help plan programs of replication studies that can provide conclusive evidence of replicability of scientific findings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.