Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research.
Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research.
复制标题
DOI：
10.1002/hbm.25154
复制
发表时间：
2022-01
影响因子：
4.8
通讯作者：
Francks C
中科院分区：
文献类型：
--
作者：
Kong XZ;ENIGMA Laterality Working Group;Francks C
近年来，科学发现的可重复性差的问题在包括心理学和神经科学在内的各个领域受到了广泛关注。这一问题部分归因于发表偏见和不受欢迎的做法，如p - hacking。个别研究的统计能力低也被认为是一个重要因素。在最近的一项多地点合作研究中，我们绘制了大脑解剖的左右不对称，以测量表面积和皮层厚度的区域测量，来自世界各地的99个MRI数据集，总共超过17,000名参与者。在本研究中，我们从可重复性的角度重新审视了这些半球效应。在每个数据集中，我们认为当一个效应在效应方向和显著性阈值方面与其他98个数据集中的元分析效应相匹配时，该效应已经被复制。从这个意义上说，每个数据集中的结果都被视为来自“理想出版环境”中的独立研究，也就是说，没有选择性报告和黑客攻击。平均重现率为63.2% （SD = 22.9%, min = 22.2%, max = 97.0%）。正如预期的那样，在更大的效应和更大的数据集中，再现性更高。重现性与受试者年龄、扫描仪场强、FreeSurfer软件版本、皮质区域测量可靠性或区域大小无明显关系。当评估异质神经科学数据中的实际生物学效应时，这些发现构成了在没有发表偏倚或p黑客的情况下可重复性的经验例证，并给出了通常使用的样本量。半球不对称效应的区域效应大小和再现率。一般来说，在相同的条件下（例如，样本量和数据异质性），效应量越大的效应具有较高的可重复性。
The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p‐hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left–right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta‐analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an “ideal publishing environment,” that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically‐used sample sizes. Region‐wise effect sizes and reproducibility rates of hemispheric asymmetry effects. In general, effects with higher effect sizes showed higher reproducibility, given the same conditions (e.g., sample size and data heterogeneity).
登录
查看更多内容
科学标准。促进开放的研究文化。
DOI：
10.1126/science.aab2374
发表时间：
2015-06-26
期刊：
Science (New York, N.Y.)
影响因子：
--
作者：
Nosek BA;Alter G;Banks GC;Borsboom D;Bowman SD;Breckler SJ;Buck S;Chambers CD;Chin G;Christensen G;Contestabile M;Dafoe A;Eich E;Freese J;Glennerster R;Goroff D;Green DP;Hesse B;Humphreys M;Ishiyama J;Karlan D;Kraut A;Lupia A;Mabry P;Madon TA;Malhotra N;Mayo-Wilson E;McNutt M;Miguel E;Paluck EL;Simonsohn U;Soderberg C;Spellman BA;Turitto J;VandenBos G;Vazire S;Wagenmakers EJ;Wilson R;Yarkoni T
通讯作者：
Yarkoni T
影响因子：
4.8
作者：
Iscan Z;Jin TB;Kendrick A;Szeglin B;Lu H;Trivedi M;Fava M;McGrath PJ;Weissman M;Kurian BT;Adams P;Weyandt S;Toups M;Carmody T;McInnis M;Cusin C;Cooper C;Oquendo MA;Parsey RV;DeLorenzo C
通讯作者：
DeLorenzo C
影响因子：
3.5
作者：
Pauli R;Bowring A;Reynolds R;Chen G;Nichols TE;Maumet C
通讯作者：
Maumet C
影响因子：
1.8
作者：
Klein, Richard A.;Ratliff, Kate A.;Nosek, Brian A.
通讯作者：
Nosek, Brian A.
影响因子：
15.8
作者：
Ioannidis, JPA
通讯作者：
Ioannidis, JPA