Hold-out validation for the assessment of stability and reliability of multivariable regression demonstrated with magnetic resonance imaging of patients with schizophrenia.
Hold-out validation for the assessment of stability and reliability of multivariable regression demonstrated with magnetic resonance imaging of patients with schizophrenia.
复制标题
通过精神分裂症患者的磁共振成像证明多变量回归的稳定性和可靠性评估的保留验证。
DOI：
10.1002/jdn.10144
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Tyrrell,Pascal
中科院分区：
文献类型：
--
作者：
Levman,Jacob;Jennings,Maxwell;Kabaria,Priya;Rouse,Ethan;Nangaku,Masahito;Berger,Derek;Gondra,Iker;Takahashi,Emi;Tyrrell,Pascal
神经科学研究的任务通常是确定两组受试者之间的可测量差异，通常是一组有病理状况，一组代表对照受试者。通常预计，为比较组而获得的测量值也会受到各种其他患者特征（如性别、年龄和合并症）的影响。多变量回归（MVR）是神经科学研究中常用的一种统计分析技术，用于“控制”或“调整”次要效应（如性别，年龄和合并症），以确保主要研究结果集中在与研究条件相关的感兴趣组之间的实际差异上。在神经科学文献中，利用MVR来控制继发效应是常见的做法;然而，目前通常无法评估MVR调整是否校正了比它们引入的更多的误差。在常见的神经科学实践中，MVR模型未经验证，也未尝试表征MVR模型中的缺陷。在这篇文章中，我们展示了如何使用标准的保持验证技术（通常用于机器学习分析），包括重复随机将数据集划分为训练和测试样本，可以适应于使用公开可用的精神分裂症患者神经磁共振成像（MRI）数据集评估MVR模型的稳定性和可靠性。结果表明，MVR可以引入高达30.06%的测量误差，并且在所有考虑的测量中，平均在该数据集上引入9.84%的误差。当有效MVR与MVR标准使用的结果不一致时，MVR在给定应用中的使用是不稳定的。因此，本文通过对精神分裂症患者的分析，帮助评估MVR的简单使用在神经科学分析中引入研究错误的程度。
Neuroscience studies are very often tasked with identifying measurable differences between two groups of subjects, typically one group with a pathological condition and one group representing control subjects. It is often expected that the measurements acquired for comparing groups are also affected by a variety of additional patient characteristics such as sex, age, and comorbidities. Multivariable regression (MVR) is a statistical analysis technique commonly employed in neuroscience studies to “control for” or “adjust for” secondary effects (such as sex, age, and comorbidities) in order to ensure that the main study findings are focused on actual differences between the groups of interest associated with the condition under investigation. It is common practice in the neuroscience literature to utilize MVR to control for secondary effects; however, at present, it is not typically possible to assess whether the MVR adjustments correct for more error than they introduce. In common neuroscience practice, MVR models are not validated and no attempt to characterize deficiencies in the MVR model is made. In this article, we demonstrate how standard hold‐out validation techniques (commonly used in machine learning analyses) that involve repeatedly randomly dividing datasets into training and testing samples can be adapted to the assessment of stability and reliability of MVR models with a publicly available neurological magnetic resonance imaging (MRI) dataset of patients with schizophrenia. Results demonstrate that MVR can introduce measurement error up to 30.06% and, on average across all considered measurements, introduce 9.84% error on this dataset. When hold‐out validated MVR does not agree with the results of the standard use of MVR, the use of MVR in the given application is unstable. Thus, this paper helps evaluate the extent to which the simplistic use of MVR introduces study error in neuroscientific analyses with an analysis of patients with schizophrenia.