Underreporting in Psychology Experiments: Evidence From a Study Registry
Underreporting in Psychology Experiments: Evidence From a Study Registry
复制标题
DOI：
10.1177/1948550615598377
复制
发表时间：
2016-01-01
影响因子：
5.7
通讯作者：
Simonovits, Gabor
中科院分区：
文献类型：
--
作者：
Franco, Annie;Malhotra, Neil;Simonovits, Gabor
许多学者对心理学实证研究结果的可信度表示担忧，认为已发表文献中报告的误报比例大大超过了标准显着性水平所暗示的比率。误报的一个主要原因是报告与研究项目相关的潜在相关统计分析子集的做法。这项研究首次提供了心理学实验中选择性漏报的直接证据。为了克服大多数已发表的研究无法获得完整的实验设计和全套测量变量的问题，我们从竞争性资助计划中确定了一组已发表的心理学实验，由于制度规则，该计划的调查问卷和数据是公开的。我们发现，约 40% 的研究未能完整报告所有实验条件，约 70% 的研究未报告调查问卷中包含的所有结果变量。报告的效应大小大约是未报告的效应大小的两倍，并且具有统计显着性的可能性大约是未报告的效应大小的三倍。
Many scholars have raised concerns about the credibility of empirical findings in psychology, arguing that the proportion of false positives reported in the published literature dramatically exceeds the rate implied by standard significance levels. A major contributor of false positives is the practice of reporting a subset of the potentially relevant statistical analyses pertaining to a research project. This study is the first to provide direct evidence of selective underreporting in psychology experiments. To overcome the problem that the complete experimental design and full set of measured variables are not accessible for most published research, we identify a population of published psychology experiments from a competitive grant program for which questionnaires and data are made publicly available because of an institutional rule. We find that about 40% of studies fail to fully report all experimental conditions and about 70% of studies do not report all outcome variables included in the questionnaire. Reported effect sizes are about twice as large as unreported effect sizes and are about 3 times more likely to be statistically significant.