How an academic system can achieve a trustworthy knowledge base: Analyzing reform proposals in an agent-based modeling approach

点击翻译按钮获取中文摘要

In several scientific fields, evidence has accumulated that the scientific literature is much less robust and trustworthy than desired, constituting a “replication crisis”. Many reforms have been proposed to increase the trustworthiness of the scientific literature, but reasonable doubts have been raised about their effectiveness, fairness, potential side-effects, and evidentiary backing. In this project, we will formalize recent arguments and reform proposals, and build agent-based models of science (where researchers are the agents) to investigate the logical coherence, efficiency, and side-effects of reforms. The models will be empirically calibrated to three disciplines from the behavioral, social, and cognitive sciences. Three specific research questions investigate the impact of reform proposals on the trustworthiness and bandwidth of scientific output: (a) Open Science as a social dilemma, or: The evolution of transparency, (b) Registered reports and a multi-layered publication system, and (c) Assessing strategies for navigating epistemic landscapes: The role of replication and theory. In a final working package (d), we aim to combine the three submodels into an integrated model. This will be used to analyze potential interactions of reforms, optimal parameter configurations, and emergent patterns arising from the interactions of the agents. We expect that the insights of this research project can inform policy development, such as journal guidelines, hiring policies of academic institutions and other incentive structures, or funder’s resource allocation strategies. Furthermore, it informs how to maximize the informational gain of scientific results, which ultimately translates to the societal utility of research.