Stimulus-specific random effects inflate false-positive classification accuracy in multivariate-voxel-pattern-analysis: A solution with generalized mixed-effects modelling
Stimulus-specific random effects inflate false-positive classification accuracy in multivariate-voxel-pattern-analysis: A solution with generalized mixed-effects modelling
复制标题
DOI：
10.1016/j.neuroimage.2023.119901
复制
发表时间：
2023-01
期刊：
影响因子：
5.7
通讯作者：
S. Kajimura;T. Hoshino;K. Murayama
中科院分区：
文献类型：
--
作者：
S. Kajimura;T. Hoshino;K. Murayama
在进行多元体素模式分析 (MVPA) 时，研究人员通常会计算每个受试者的平均准确度，并统计测试平均准确度是否与受试者之间的机会水平不同（按受试者分析）。我们认为，无论使用哪种类型的机器学习方法（例如支持向量机），这种传统的按主题分析都会导致 1 类错误率过高。这是因为按受试者分析没有考虑对所有受试者有共同影响的刺激的特殊特征所引起的方差（即随机刺激效应）。作为解决方案，我们提出使用广义线性混合效应模型来评估平均精度。该方法只需要分类后的数据（即不考虑所使用的分类方法的类型），并且很容易在常用统计软件（SPSS、R、Python等）的分析流程中实现。使用统计模拟和真实的功能磁共振成像数据分析，我们证明了传统的按受试者方法确实在相当程度上增加了 1 类错误率，而纳入随机刺激效应的广义混合效应模型确实可以保持名义 1 类错误率。
When conducting multivariate-voxel pattern analysis (MVPA), researchers typically compute the average accuracy for each subject and statistically test if the average accuracy is different from the chance level across subjects (by-subject analysis). We argue that this traditional by-subject analysis leads to inflated Type-1 error rates, regardless of the type of machine learning method used (e.g., support vector machine). This is because by-subject analysis does not consider the variance attributed to the idiosyncratic features of the stimuli that have a common influence on all subjects (i.e., the random stimulus effect). As a solution, we proposed the use of generalized linear mixed-effects modelling to evaluate average accuracy. This method only requires post-classification data (i.e., it does not consider the type of classification methods used) and is easily implemented in the analysis pipeline with common statistical software (SPSS, R, Python, etc.). Using both statistical simulation and real fMRI data analysis, we demonstrated that the traditional by-subject method indeed increases Type-1 error rates to a considerable degree, while generalized mixed-effects modelling that incorporates random stimulus effects can indeed maintain the nominal Type-1 error rates.