Methods for assessing replication

先前发现和结果的复制是科学的一个基本特征，也是支持科学具有自我纠正能力这一主张的逻辑的一部分。然而，很少有以前的研究方法研究复制。 涉及荟萃分析和系统评价的研究，总结了一系列研究报告，更为常见。然而，一组实验研究的结果是否相互复制的问题却受到较少的关注。 对于成功的重复研究或统计文献，没有明确定义和广泛接受的定义，也没有提供如何设计单次重复研究或一组重复研究的方法学指南。这里提出的研究建立了这种急需的方法。该项目由Discovery Research PreK-12计划资助，该计划资助STEM创新和方法的研究和开发。该项目的目标是正式确定关于复制重要概念的主观想法，为评估复制研究提供统计分析，为评估复制研究的结论性提供属性，并为设计结论性和有效的复制研究方案提供了原则。它涉及三个基本问题。第一个问题是如何定义重复性：一组研究的结果相互重复，这究竟意味着什么？ 第二，给出重复的定义，应该做什么统计分析来决定研究的集合是否相互重复，以及这些分析的属性是什么（例如，敏感性或统计功效）？ 第三，如何设计一项或多项复制研究，为复制问题提供结论性答案？该项目通过提供统计工具来评估实验结果的可复制性，评估复制尝试的结论性，和开发软件，以帮助计划的复制研究，可以提供科学发现的可复制性的决定性证据。这个奖项反映了NSF的法定使命，并已被认为是值得支持，通过评估使用基金会的学术价值和更广泛的影响评审标准。

Replication of prior findings and results is a fundamental feature of science and is part of the logic supporting the claim that science is self-correcting. However, there is little prior research on the methodology for studying replication. Research involving meta-analysis and systematic reviews that summarizes a collection of research studies is more common. However, the question of whether the findings from a set of experimental studies replicate one another has received less attention. There is no clearly defined and widely accepted definition of a successful replication study or statistical literature providing methodological guidelines on how to design single replication studies or a set of replication studies. The research proposed here builds this much needed methodology. This project is funded by the Discovery Research PreK-12 Program, which funds research and development on STEM innovations and approaches.The goal of this project is to formalize subjective ideas about the important concept of replication, provide statistical analyses for evaluating replication studies, provide properties for evaluating the conclusiveness of replication studies, and provide principles for designing conclusive and efficient programs of replication studies. It addresses three fundamental problems. The first is how to define replication: What, precisely, should it mean to say that the results in a collection of studies replicate one another? Second, given a definition of replication, what statistical analyses should be done to decide whether the collection of studies replicate one another and what are the properties of these analyses (e.g., sensitivity or statistical power)? Third, how should one or more replication studies be designed to provide conclusive answers to questions of replication? The project has the potential for impact on a range of empirical sciences by providing statistical tools to evaluate the replicability of experimental findings, assessing the conclusiveness of replication attempts, and developing software to help plan programs of replication studies that can provide conclusive evidence of replicability of scientific findings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.