Opening the File Drawer: Assessing and Understanding Publication Bias in the Economic, Behavioral and Social Sciences by Utilizing two German Academic Access Panels

发表偏差可以被认为是由于研究方向或强度的原因而导致的研究成果的发表(或不发表)。例如，当统计上有意义的结果比统计上不显著的结果更有可能被公布时。因此，发表偏见阻碍了对特定研究问题知识真实状况的评估，导致基于虚假或不充分的科学证据的科学辩论和政治决定。有几个因素影响着出版偏见的出现，最重要的是直接涉及的参与者：作者、编辑和审稿人。此外，学科和文化背景也影响发表偏见的可能性和程度。在我们的项目中，我们计划将重点放在研究人员决策过程的各个步骤上，这可能会导致选定结果的发布，而将其他结果留在“文件抽屉”中。该项目有两个主要目标：(I)首先，它旨在衡量德国经济、行为和社会科学发表偏见的程度。这将通过对比已发表和未发表的GEIS专家组和SOEP-创新样本(SOEP-IS)研究提交的材料来实现。(Ii)第二，在现有的有关出版偏差成因的理论框架的基础上，我们将对所得出的假说进行实证检验，并(如有必要)调整和/或扩展现有的理论模型。为了调查研究人员的决策过程，这个项目将处于独特的位置，打开文件抽屉，观察整个研究周期。它的目的是通过使用外部研究人员向全球环境信息系统小组和国家环境政策信息系统提交的研究报告中的数据，密切关注从研究建议到出版阶段的研究做法。在这两个德国接入面板基础设施中，来自所有学科的外部研究人员可以提交自己的研究报告。然而，它们必须包括对理论背景、假设和研究设计的全面描述，然后接受同行审查。大多数成功的研究提交都会导致相应的发表；然而，有些则不会。在这种情况下，来自两个小组的184份成功的研究报告使我们能够调查哪些研究特征(例如，资历和作者数量，试验性和非试验性研究设计)与成功发表的可能性更高。此外，我们将进行一项作者调查，以收集关于研究过程中动机、态度和价值观的更多基于演员的数据。此外，对于随机选择的没有任何相应出版物的成功研究提交的子集，我们将使用Gesis Panel和SOEP-IS数据进行自己的统计分析，以调查它们是否有很高的可能性产生统计上不显著的结果，这可能解释为什么没有相应的出版物。

Publication bias can be regarded as the publication (or non-publication) of study results due to the direction or strength. That is when, for example, statistically significant results are more likely to be published than statistically non-significant results. Consequently, publication bias hinders the assessment of the true state of knowledge on a particular research question, resulting in scientific debates and political decisions that are based on false or inadequate scientific evidence. Several factors impact the emergence of publication bias, most importantly the actors directly involved: authors, editors, and reviewers. In addition, the disciplinary and cultural background affects the likelihood and extent of publication bias as well. In our project, we plan to focus on the various steps of researchers’ decision-making processes, which may lead to the publication of selected results while leaving other results in the “file drawer.” The project has two main objectives: (I) First, it aims to measure the extent of publication bias for the economic, behavioral, and social sciences in Germany. This will be accomplished by contrasting published and unpublished GESIS Panel and SOEP-Innovation Sample (SOEP-IS) study submissions. (II) Second, building upon existing theoretical frameworks on the causes of publication bias, we will empirically test the derived hypotheses and (if necessary) adjust and/or extent the current theoretical models. To investigate researchers’ decision processes, this project will be in the unique position to open the file drawer and observe the entire research cycle. It aims to look closely at research practices during the way from a study proposal to the publication stage by using data from study submissions of external researchers to the GESIS Panel and the SOEP-IS. In these two German access panel infrastructures, external researchers from all disciplines can submit their own studies. However, they must include a thorough description of the theoretical background, the hypotheses, the research design, which are then subject to a peer review. Most successful study submissions result in a corresponding publication; some, however, do not. In this context, 184 successful study submissions from both panels combined allow us to investigate which study characteristics (e.g., seniority and number of authors, experimental vs. non-experimental research design) are associated with a higher likelihood of a successful publication. In addition, we will conduct an author survey to collect further actor-based data on motives, attitudes, and values with respect to the research process. Furthermore, for a randomly selected subset of successful study submissions without any corresponding publication, we will conduct own statistical analyses using GESIS Panel and SOEP-IS data to investigate if they have a high likelihood of producing statistically non-significant results, which might explain the lack of a corresponding publication.