Opening the File Drawer: Assessing and Understanding Publication Bias in the Economic, Behavioral and Social Sciences by Utilizing two German Academic Access Panels

打开文件抽屉：利用两个德国学术访问面板评估和理解经济、行为和社会科学领域的出版偏见

• 批准号: 512014619
• 负责人: Dr. Jessica Daikeler
• 金额: --
• 依托单位: Institute for Policy Research
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/512014619?language=en
• 关键词: Opening; File; Drawer; Assessing; Understanding

Publication bias can be regarded as the publication (or non-publication) of study results due to the direction or strength. That is when, for example, statistically significant results are more likely to be published than statistically non-significant results. Consequently, publication bias hinders the assessment of the true state of knowledge on a particular research question, resulting in scientific debates and political decisions that are based on false or inadequate scientific evidence. Several factors impact the emergence of publication bias, most importantly the actors directly involved: authors, editors, and reviewers. In addition, the disciplinary and cultural background affects the likelihood and extent of publication bias as well. In our project, we plan to focus on the various steps of researchers’ decision-making processes, which may lead to the publication of selected results while leaving other results in the “file drawer.” The project has two main objectives: (I) First, it aims to measure the extent of publication bias for the economic, behavioral, and social sciences in Germany. This will be accomplished by contrasting published and unpublished GESIS Panel and SOEP-Innovation Sample (SOEP-IS) study submissions. (II) Second, building upon existing theoretical frameworks on the causes of publication bias, we will empirically test the derived hypotheses and (if necessary) adjust and/or extent the current theoretical models. To investigate researchers’ decision processes, this project will be in the unique position to open the file drawer and observe the entire research cycle. It aims to look closely at research practices during the way from a study proposal to the publication stage by using data from study submissions of external researchers to the GESIS Panel and the SOEP-IS. In these two German access panel infrastructures, external researchers from all disciplines can submit their own studies. However, they must include a thorough description of the theoretical background, the hypotheses, the research design, which are then subject to a peer review. Most successful study submissions result in a corresponding publication; some, however, do not. In this context, 184 successful study submissions from both panels combined allow us to investigate which study characteristics (e.g., seniority and number of authors, experimental vs. non-experimental research design) are associated with a higher likelihood of a successful publication. In addition, we will conduct an author survey to collect further actor-based data on motives, attitudes, and values with respect to the research process. Furthermore, for a randomly selected subset of successful study submissions without any corresponding publication, we will conduct own statistical analyses using GESIS Panel and SOEP-IS data to investigate if they have a high likelihood of producing statistically non-significant results, which might explain the lack of a corresponding publication.

发表偏倚是指由于研究的方向或力度而导致研究结果的发表（或未发表）。例如，当统计上显著的结果比统计上不显著的结果更有可能被发表时。因此，发表偏见阻碍了对特定研究问题的真实知识状况的评估，导致基于虚假或不充分科学证据的科学辩论和政治决策。有几个因素会影响出版偏见的出现，最重要的是直接参与者：作者，编辑和审稿人。此外，学科和文化背景也会影响发表偏倚的可能性和程度。在我们的项目中，我们计划关注研究人员决策过程的各个步骤，这可能会导致选定的结果发表，而将其他结果留在“文件抽屉”中。该项目有两个主要目标：（1）首先，它旨在衡量经济，行为和社会科学在德国的出版偏见的程度。这将通过对比已发表和未发表的GESIS小组和SOEP创新样本（SOEP-IS）研究提交来完成。(II)第二，建立在现有的理论框架的原因发表偏见，我们将实证检验所得出的假设和（如有必要）调整和/或扩展现有的理论模型。为了调查研究人员的决策过程，这个项目将处于独特的位置，打开文件抽屉，观察整个研究周期。它旨在通过使用外部研究人员向GESIS小组和SOEP-IS提交的研究报告中的数据，密切关注从研究提案到出版阶段的研究实践。在这两个德国接入面板基础设施中，来自所有学科的外部研究人员都可以提交自己的研究。然而，它们必须包括对理论背景、假设和研究设计的全面描述，然后接受同行审查。大多数成功提交的研究报告都会发表相应的文章，但也有一些没有。在这种情况下，来自两个小组的184项成功的研究提交使我们能够调查哪些研究特征（例如，资历和作者人数，实验与非实验研究设计）与成功发表的可能性较高有关。此外，我们还将进行一项作者调查，以进一步收集有关研究过程中动机、态度和价值观的基于行为者的数据。此外，对于随机选择的成功研究提交的子集，没有任何相应的出版物，我们将使用GESIS Panel和SOEP-IS数据进行自己的统计分析，以调查它们是否有很高的可能性产生统计上不显著的结果，这可能解释缺乏相应的出版物。