Improving Science Communication via Results-Blind Selection: Lessons from Registered Reports

我们经常被告知，科学是值得信赖的，因为科学家关注的是如何回答一个问题，而不是决定答案是什么。例如，一位心理科学家可能会精心设计一项研究来比较严格和宽松的教学风格，这可能会让答案出乎意料，甚至令人失望。然而，与科学家不同的是，公众了解的答案是由科学记者和编辑选择的，这一系统为不信任打开了大门。这种不信任在皮尤最近的一份报告中表现得很明显，该报告发现，只有52%的美国人认为媒体在报道科学方面做得很好，这一比例在接受调查的20个国家中排名倒数第二。如果记者更有可能报道耸人听闻的发现，而不是那些有有力证据支持的发现，这可能会损害这些发现作为指导来源的可靠性。此外，如果记者更有可能报道强化某种特定意识形态的发现，这可能会助长两极分化和公众的不信任。目前的提案将测试一种提高心理科学报道可信度的策略：记者可以根据方法而不是结果来选择研究（结果盲选择）。因此，这种方法将重点转移到优先考虑声音，而不是耸人听闻的发现。通过追求提高科学新闻可信度的具体策略，该提案与美国国家科学基金会“增加科学活动的公共价值”的目标一致。结果盲评估可以减少偏倚，提高特色研究的质量，这一理念是注册报告学术文章格式的基础，该格式于2013年在Cortex首次推出。先前的研究表明，注册报告在严谨性和质量上的评分高于比较论文，这表明结果盲选择可能是提高质量和可复制性的有效方法。在本项目中，我们提出三项研究。在研究1中，我们将招募400名记者，测试科学研究的结果盲选择是否可以提高报道研究的可信度。在研究2中，我们将招募250名参与者，测试科学研究的结果盲选择是否可以增加公众对科学新闻的信任。最后，在研究3中，我们将重新联系研究1中的32名记者，并使用小组访谈来探索如何将结果盲选择有效地与现有的新闻规范和惯例相结合。这些发现将使我们了解依赖于结果盲选择的现有机制——比如注册报告——以及我们对这种方法的潜在价值的理解，这种方法可以提高心理科学的质量，并向公众传播。该项目由“科学之科学：发现、传播和影响”和“推进非正式STEM学习计划”共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

We are often told that science is trustworthy because scientists focus on how to answer a question without getting to decide what that answer will be. For instance, a psychological scientist might carefully design a study to compare strict and lenient teaching styles, opening up the possibility that the answer could be unexpected, or even disappointing. Unlike scientists, however, the public learns the answers that have been selected by science journalists and editors, a system that opens the door to distrust. This distrust is apparent in a recent PEW report, which found that only 52% of Americans say that the media do a good job of covering science, a rate that puts the US second-to-last among the 20 countries surveyed. If journalists are more likely to report on sensationalistic findings over those supported by strong evidence, this could damage the dependability of those findings as source of guidance. Further, if journalists are more likely to report findings that reinforce a particular ideology, this could foster polarization and public distrust. The present proposal will test one strategy for improving the trustworthiness of psychological science reporting: journalists could select research based on methods rather than results (results-blind selection). This approach thus shifts the focus to prioritizing sound, rather than sensational, findings. By pursuing concrete strategies for improving the trustworthiness of science journalism, this proposal aligns with the National Science Foundation’s goal to “increase the public value of scientific activity.” The idea that results-blind evaluation could reduce bias and improve the quality of featured research is the foundation for the Registered Reports academic article format, first introduced at Cortex in 2013. Previous research shows that Registered Reports are rated higher than the comparison papers on rigorousness and quality, suggesting that results-blind selection may be an effective way to promote quality and replicability. In the present project, we propose three studies. In Study 1, we will recruit 400 journalists and test whether results-blind selection of scientific studies can improve the trustworthiness of reported research. In Study 2, we will recruit 250 participants and test whether results-blind selection of scientific studies can increase public trust in science journalism. Finally, in Study 3 we will re-contact 32 journalists from Study 1 and use group interviews to explore how results-blind selection could be effectively integrated with existing journalistic norms and routines. These findings will inform our understanding of existing mechanisms that rely on results-blind selection – like Registered Reports – as well as our understanding of the potential value of this approach to improving the quality the psychological science that gets communicated to the public.The project is co-funded by the Science of Science: Discovery, Communications, and Impact and the Advancing Informal STEM Learning Programs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.