Improving Science Communication via Results-Blind Selection: Lessons from Registered Reports

我们经常被告知，科学是值得信赖的，因为科学家专注于如何回答一个问题，而不是决定答案是什么。例如，一位心理科学家可能会精心设计一项研究，比较严格和宽松的教学风格，从而打开了答案可能出乎意料甚至令人失望的可能性。然而，与科学家不同的是，公众了解的是科学记者和编辑选择的答案，这一系统为不信任打开了大门。这种不信任在皮尤最近的一份报告中显而易见，该报告发现，只有52%的美国人认为媒体在报道科学方面做得很好，这一比例使美国在接受调查的20个国家中排名倒数第二。如果记者更倾向于报道耸人听闻的调查结果，而不是有强有力证据支持的调查结果，这可能会损害这些调查结果作为指导来源的可靠性。此外，如果记者更有可能报道强化特定意识形态的调查结果，这可能会助长两极分化和公众不信任。目前的建议将测试一种提高心理科学报道可信度的策略：记者可以根据方法而不是结果选择研究（结果盲选）。因此，这种方法将重点转移到优先考虑合理的，而不是耸人听闻的调查结果。通过追求提高科学新闻可信度的具体策略，该提案符合美国国家科学基金会的目标，即“增加科学活动的公共价值”。结果盲法评估可以减少偏见并提高特色研究的质量，这一想法是2013年在Cortex首次推出的Registered Reports学术文章格式的基础。以往的研究表明，注册报告的严谨性和质量的评价高于比较论文，这表明结果盲选可能是一种有效的方法，以提高质量和可复制性。在本项目中，我们提出了三项研究。在研究1中，我们将招募400名记者，测试科学研究的结果盲选是否可以提高报道研究的可信度。在研究2中，我们将招募250名参与者，并测试科学研究的结果盲选是否可以增加公众对科学新闻的信任。最后，在研究3中，我们将重新接触研究1中的32名记者，并使用小组访谈来探索如何将结果盲选与现有的新闻规范和惯例有效地结合起来。这些发现将使我们了解依赖于结果盲选的现有机制（如注册报告），以及我们了解这种方法对提高向公众传播的心理科学质量的潜在价值。该项目由科学科学共同资助：发现，通信，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值进行评估来支持和更广泛的影响审查标准。

We are often told that science is trustworthy because scientists focus on how to answer a question without getting to decide what that answer will be. For instance, a psychological scientist might carefully design a study to compare strict and lenient teaching styles, opening up the possibility that the answer could be unexpected, or even disappointing. Unlike scientists, however, the public learns the answers that have been selected by science journalists and editors, a system that opens the door to distrust. This distrust is apparent in a recent PEW report, which found that only 52% of Americans say that the media do a good job of covering science, a rate that puts the US second-to-last among the 20 countries surveyed. If journalists are more likely to report on sensationalistic findings over those supported by strong evidence, this could damage the dependability of those findings as source of guidance. Further, if journalists are more likely to report findings that reinforce a particular ideology, this could foster polarization and public distrust. The present proposal will test one strategy for improving the trustworthiness of psychological science reporting: journalists could select research based on methods rather than results (results-blind selection). This approach thus shifts the focus to prioritizing sound, rather than sensational, findings. By pursuing concrete strategies for improving the trustworthiness of science journalism, this proposal aligns with the National Science Foundation’s goal to “increase the public value of scientific activity.” The idea that results-blind evaluation could reduce bias and improve the quality of featured research is the foundation for the Registered Reports academic article format, first introduced at Cortex in 2013. Previous research shows that Registered Reports are rated higher than the comparison papers on rigorousness and quality, suggesting that results-blind selection may be an effective way to promote quality and replicability. In the present project, we propose three studies. In Study 1, we will recruit 400 journalists and test whether results-blind selection of scientific studies can improve the trustworthiness of reported research. In Study 2, we will recruit 250 participants and test whether results-blind selection of scientific studies can increase public trust in science journalism. Finally, in Study 3 we will re-contact 32 journalists from Study 1 and use group interviews to explore how results-blind selection could be effectively integrated with existing journalistic norms and routines. These findings will inform our understanding of existing mechanisms that rely on results-blind selection – like Registered Reports – as well as our understanding of the potential value of this approach to improving the quality the psychological science that gets communicated to the public.The project is co-funded by the Science of Science: Discovery, Communications, and Impact and the Advancing Informal STEM Learning Programs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.