NSF PD 19-125Y - Science of Science: Discovery, Communication, and Impact (SoS:DCI) -Randomized Control Trial of the Registered Reports Publishing Format

注册报告（RR）出版模式可以改变研究的进行，同行评审和出版方式。RR可能会改变研究人员的激励机制，使他们专注于提出重要的问题，并使用强有力的方法来测试这些问题，而不再需要令人兴奋的、具有煽动性的结果，无论它们的可信度如何。最终，RR可以提高同行评审和学术交流的质量，从而提高研究的严谨性，透明度和可重复性。大规模采用RR可能会从根本上改变研究的评估和沟通方式。在RR中，作者提交一篇论文，详细说明研究问题，并在观察研究结果之前提出测试问题的方法。在同行评议的第一阶段，审稿人会评估问题的重要性和研究方法的质量。如果论文通过了这一阶段的同行评议，作者将获得“原则上接受”，这意味着只要作者能够胜任研究的执行和报告，期刊就承诺发表论文，而不管结果如何。同行评议的第二阶段，在结果已知并添加到论文中之后，评估是否遵守最初的承诺，计划和计划外分析之间的区别是否清晰，以及对结果解释的准确性，而不是结果是否积极，有趣或与假设一致。考虑到RR对研究过程的潜在变革性影响，进行随机试验以评估模型的质量至关重要。通过与期刊合作，我们将在生态有效的背景下对RR进行随机试验。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The Registered Reports (RR) publishing model could be transformative to how research is conducted, peer reviewed, and published. RRs could alter incentives for researchers toward focusing on asking important questions and using strong methodology to test those questions, and away from the demand for exciting, provocative results whatever their credibility. Ultimately, RRs can improve the quality of peer review and scholarly communication, thus improving the rigor, transparency, and reproducibility of research. Large-scale adoption of RRs could fundamentally change how research is evaluated and communicated.In a RR, authors submit a paper detailing the research question and proposed methodology to test the question prior to observing the study outcomes. In the first stage of peer review, reviewers assess the importance of the question and the quality of the methodology proposed to investigate it. If the paper passes this stage of peer review, the authors receive “in-principle acceptance” meaning that the journal commits to publishing the paper regardless of outcomes as long as the authors follow through with competent execution and reporting of the research. The second stage of peer review, after the results are known and added to the paper, assesses adherence to the original commitments, clarity of the distinction between planned and unplanned analyses, and accuracy of interpretation of findings, and not whether the results are positive, interesting, or consistent with hypotheses. Given the potentially transformative impact of RRs on the research process, it is essential to conduct a randomized trial to evaluate the model’s qualities. By collaborating with journals, we will conduct a randomized trial of RRs in an ecologically valid context.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.