Power contours: Optimising sample size and precision in experimental psychology and human neuroscience.
Power contours: Optimising sample size and precision in experimental psychology and human neuroscience.
复制标题
Power contours：Optimizing sample size and precision in experimental psychology and human neuroscience.
DOI：
10.1037/met0000337
复制
发表时间：
2021-06
影响因子：
7
通讯作者：
Andrews TJ
中科院分区：
文献类型：
--
作者：
Baker DH;Vilidaite G;Lygo FA;Smith AK;Flack TR;Gouws AD;Andrews TJ
当设计有人类参与者的实验研究时，实验者必须决定每个参与者将完成多少次试验，以及要测试多少参与者。大多数关于统计能力（研究设计检测效果的能力）的讨论都集中在样本量上，并假设有足够的试验。在这里，我们探讨了这两个因素对统计功率的影响，用二维图表示，在二维图上可以可视化等功率轮廓。我们论证了试验数特别重要的条件，即当参与者内方差相对于参与者间方差较大时。然后，我们使用8种实验范式和方法（包括反应时间、感觉阈值、fMRI、MEG和EEG）的现有数据集得出功率等高线图，并提供示例代码来计算每种方法的参与者内部和参与者之间方差的估计值。在所有情况下，参与者内方差大于参与者间方差，这意味着试验数量对常用范式的统计能力有显著影响。提供了一个在线工具（https://shiny.york.ac.uk/powercontours/）用于生成功率轮廓，在设计未来的研究时，可以从中计算出试验和参与者的最佳组合。神经科学和实验心理学的许多研究都涉及在给定条件下对人类参与者进行多次测试，并在这些重复中取平均值，以更准确地估计真实反应。然而，大多数研究人员没有一个有原则的方法来决定他们应该进行多少试验，而且决定往往是用武断的标准做出的。这是一个重要的问题，因为试验的数量对研究的统计能力有直接的影响——它能够检测到实际效果的可能性。在最近的心理学“复制危机”的背景下，研究人员需要工具来优化他们的研究设计的质量，以增加权力。在这里，我们提出了一种方法来可视化样本量（测试参与者的数量）和每个参与者的试验数量对统计能力的综合影响，使用二维等高线图。我们从一系列广泛使用的方法（包括反应时间、脑电图、脑磁图和功能磁共振成像）中对八个现有数据集进行了抽样，结果表明这些轮廓是弯曲的，并允许在研究设计阶段估计参与者和试验的最佳数量。我们提供了所有的分析脚本，以及一个在线工具，以允许其他人根据他们自己的实验范例来调整我们的方法。我们预计这种方法将有助于设计更有效的实验研究，更有可能报告实际效果。
When designing experimental studies with human participants, experimenters must decide how many trials each participant will complete, as well as how many participants to test. Most discussion of statistical power (the ability of a study design to detect an effect) has focused on sample size, and assumed sufficient trials. Here we explore the influence of both factors on statistical power, represented as a 2-dimensional plot on which iso-power contours can be visualized. We demonstrate the conditions under which the number of trials is particularly important, that is, when the within-participant variance is large relative to the between-participants variance. We then derive power contour plots using existing data sets for 8 experimental paradigms and methodologies (including reaction times, sensory thresholds, fMRI, MEG, and EEG), and provide example code to calculate estimates of the within- and between-participants variance for each method. In all cases, the within-participant variance was larger than the between-participants variance, meaning that the number of trials has a meaningful influence on statistical power in commonly used paradigms. An online tool is provided (https://shiny.york.ac.uk/powercontours/) for generating power contours, from which the optimal combination of trials and participants can be calculated when designing future studies. Many studies in neuroscience and experimental psychology involve testing human participants multiple times in a given condition, and averaging across these repetitions to get a more accurate estimate of the true response. Yet most researchers do not have a principled way to decide how many trials they should conduct, and decisions are often made using arbitrary criteria. This is an important issue because the number of trials has a direct effect on the statistical power of a study—the likelihood that it is able to detect a real effect. In the context of the recent “replication crisis” in psychology, researchers need tools to optimize the quality of their research designs to increase power. Here we propose a way to visualize the combined effect of sample size (the number of participants tested) and number of trials per participant on statistical power, using a two-dimensional contour plot. We show by subsampling eight existing data sets from a range of widely used methods (including reaction times, EEG, MEG, and fMRI) that these contours are curved, and permit estimation of an optimal number of participants and trials at the study design stage. All of the analysis scripts, as well as an online tool, are provided to permit others to tailor our methods to their own experimental paradigms. We anticipate that this approach will facilitate the design of experimental studies that are more efficient, and more likely to report real effects.
登录
查看更多内容
影响因子：
3.6
作者：
Flack, Tessa R.;Andrews, Timothy J.;Young, Andrew W.
通讯作者：
Young, Andrew W.
影响因子：
3.5
作者：
Rouder, Jeffrey N.;Speckman, Paul L.;Iverson, Geoffrey
通讯作者：
Iverson, Geoffrey
影响因子：
15.8
作者：
Ioannidis, JPA
通讯作者：
Ioannidis, JPA
影响因子：
3.8
作者：
Brandmaier AM;von Oertzen T;Ghisletta P;Hertzog C;Lindenberger U
通讯作者：
Lindenberger U
影响因子：
3.5
作者：
Colquhoun D
通讯作者：
Colquhoun D