METEOR – Mastering the oppressive number of forking paths unfolded by noisy and complex neural data

在心理学和认知神经科学中存在着复制危机和“现实世界还是实验室”的困境。在解决困境的同时克服危机可以说是一项严峻的挑战。认知神经科学的主要目标之一是发现可在实验室中复制的脑-认知关联。在实验室内外获得的脑-认知关联方面的个体差异发现的可复制性的先决条件是来自噪声和复杂信号记录的神经参数的秩序稳定性。然而，到目前为止，我们还不清楚有多少迄今为止不成功的复制是由于研究人员在测试大脑-认知关联时必须<s:1>先验地做出大量方法决定。此外，我们还没有关于分析单位的标准，在这个单位上，复制应该被认为是成功的。我们还缺乏一个知识应用程序，该应用程序包含系统和详尽的潜在方法选择概述，这些方法选择在典型的移动脑电图或功能磁共振成像的个体差异分析工作流程中是可防御的，以及多变量行为数据。因此，实验室仍然坚持他们的定制选择，而这些选择有时被年轻科学家代代相传。但是，正如最近的文献所描述的那样，工作流程的可变性和相关的实质性发现在实验室中是巨大的。最后，迄今提出的用于分析嘈杂和高度复杂的多维神经数据的潜在构建数据集的多元宇宙的统计方法需要通过可用于大数据分析的工具进行扩展。这种方法将有助于了解有影响的决定，并预测未来研究结果的潜在异质性。为了填补这些空白，METEOR的目标是汇集更多具有不同和互补专业知识的科学家（使用移动脑电图方法的认知神经科学家，使用功能磁共振成像数据的网络神经科学家和经验丰富的大数据分析工具的统计学家）。通过联合力量和富有成效的合作研究项目环境，METEOR将为适用于神经成像模式的认知神经科学的复制成功定义提供标准。此外，它将提供系统化的知识和多元宇宙的分析解决方案，在两种神经成像模式-移动脑电图和静息状态功能磁共振成像-应用于评估个体差异和大脑认知关联的领域。提出的解决方案将讨论其对未来进一步研究问题的适用性。

There is a replication crisis and a “real-world or the lab” dilemma in psychology and cognitive neuroscience. Solving the dilemma and overcoming the crisis at the same time is arguably a serious challenge. One of the main aims in cognitive neuroscience is to discover brain-cognition associations which are replicable across laboratories. A precondition for replicability of individual differences findings in terms of brain-cognition associations obtained inside or outside of the laboratory is rank order stability of neural parameters derived from noisy and complex signal recordings. However, to date we do not know well enough how much hitherto unsuccessful replications are due to the oppressive number of methodological decisions researchers have to make á priori to testing a brain-cognition association. Moreover, we do not yet have standards with respect to the unit of analysis at which replications should be considered successful. We also lack a knowledge app containing a systematic and exhaustive overview of potential methodological choices that are defensible in a typical individual differences analysis workflow for mobile EEG or fMRI, as well as multivariate behavioral data. Thus, laboratories still stick to their customized choices which are sometimes passed over through many generations of young scientist. But – as described in the very recent literature – variability of workflows and of associated substantial findings is huge across laboratories. Finally, hitherto proposed statistical approaches for analyzing the multiverse of potentially constructed datasets for noisy and highly complex multidimensional neural data need extensions through tools available for big data analysis. Such approaches would allow learning about influential decisions and would predict potential heterogeneity of future findings. To take a large step toward filling these gaps, METEOR aims to bring together a larger group of scientists with different and complementary expertise (cognitive neuroscientists using mobile EEG methodology, network neuroscientists working with fMRI data and statisticians experienced with big data analyses tools). By joining forces and a fruitful environment of a collaborative research programme, METEOR will provide standards on a replication success definition for cognitive neuroscience applicable across neuroimaging modalities. Furthermore, it will deliver systematized knowledge and analytic solutions for the multiverse in two neuroimaging modalities – mobile EEG and resting state fMRI – applied to the realm of assessiong individual differences and brain cognition associations. Proposed solutions will be discussed with respect to their applicability to further research questions in the future.