How an academic system can achieve a trustworthy knowledge base: Analyzing reform proposals in an agent-based modeling approach

在一些科学领域，越来越多的证据表明，科学文献远不如人们所期望的那样健全和可信，从而构成了“复制危机”。为了提高科学文献的可信度，已经提出了许多改革建议，但人们对这些改革的有效性、公平性、潜在的副作用和证据支持提出了合理的质疑。在这个项目中，我们将正式化最近的争论和改革建议，并建立基于代理人的科学模型（其中研究人员是代理人），以调查改革的逻辑一致性，效率和副作用。这些模型将根据行为、社会和认知科学的三个学科进行经验校准。三个具体的研究问题调查改革建议对科学产出的可信度和带宽的影响：（a）开放科学作为一个社会困境，或：透明度的演变，（B）注册报告和多层出版系统，（c）评估策略导航认知景观：复制和理论的作用。在最后的工作包（d）中，我们的目标是将三个子模型联合收割机组合成一个集成模型。这将被用来分析潜在的相互作用的改革，最佳参数配置，以及从代理人的相互作用所产生的紧急模式。我们希望本研究项目的见解可以为政策制定提供信息，例如期刊指南，学术机构的招聘政策和其他激励结构，或资助者的资源分配策略。此外，它告知如何最大限度地提高科学成果的信息增益，最终转化为研究的社会效用。

In several scientific fields, evidence has accumulated that the scientific literature is much less robust and trustworthy than desired, constituting a “replication crisis”. Many reforms have been proposed to increase the trustworthiness of the scientific literature, but reasonable doubts have been raised about their effectiveness, fairness, potential side-effects, and evidentiary backing. In this project, we will formalize recent arguments and reform proposals, and build agent-based models of science (where researchers are the agents) to investigate the logical coherence, efficiency, and side-effects of reforms. The models will be empirically calibrated to three disciplines from the behavioral, social, and cognitive sciences. Three specific research questions investigate the impact of reform proposals on the trustworthiness and bandwidth of scientific output: (a) Open Science as a social dilemma, or: The evolution of transparency, (b) Registered reports and a multi-layered publication system, and (c) Assessing strategies for navigating epistemic landscapes: The role of replication and theory. In a final working package (d), we aim to combine the three submodels into an integrated model. This will be used to analyze potential interactions of reforms, optimal parameter configurations, and emergent patterns arising from the interactions of the agents. We expect that the insights of this research project can inform policy development, such as journal guidelines, hiring policies of academic institutions and other incentive structures, or funder’s resource allocation strategies. Furthermore, it informs how to maximize the informational gain of scientific results, which ultimately translates to the societal utility of research.