Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines
Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines
复制标题
DOI：
10.1016/j.neuroimage.2016.10.038
复制
发表时间：
2017-01-15
期刊：
影响因子：
5.7
通讯作者：
Thirion, Bertrand
中科院分区：
文献类型：
--
作者：
Varoquaux, Gael;Raamana, Pradeep Reddy;Thirion, Bertrand
解码，即根据大脑图像或信号进行预测，需要对其预测能力进行实证评估。这种评估是通过交叉验证来实现的，这种方法也用于调整解码器的超参数。本文是对神经影像解码交叉验证程序的综述。它包括相关理论考虑的教学概述。通过对受试者内和跨受试者预测中的常见解码器、解剖和功能 MRI 和 MEG 和模拟的多个数据集进行广泛的实证研究，突出了实际方面。理论和实验表明，流行的“留一法”策略会导致估计不稳定且有偏差，因此应首选重复随机分割方法。实验概述了神经影像学设置中交叉验证的大误差线：典型的置信区间为 10%。嵌套交叉验证可以调整解码器的参数，同时避免循环偏差。然而，我们发现使用合理的默认值可能是有利的，特别是对于非稀疏解码器。
Decoding, i.e. prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross-validation, a method also used to tune decoders' hyper-parameters. This paper is a review on cross-validation procedures for decoding in neuroimaging. It includes a didactic overview of the relevant theoretical considerations. Practical aspects are highlighted with an extensive empirical study of the common decoders in within- and across-subject predictions, on multiple datasets anatomical and functional MRI and MEG- and simulations. Theory and experiments outline that the popular "leave-one-out" strategy leads to unstable and biased estimates, and a repeated random splits method should be preferred. Experiments outline the large error bars of cross-validation in neuroimaging settings: typical confidence intervals of 10%. Nested cross-validation can tune decoders' parameters while avoiding circularity bias. However we find that it can be favorable to use sane defaults, in particular for non-sparse decoders.