Inattentive responding can induce spurious associations between task behavior and symptom measures
Inattentive responding can induce spurious associations between task behavior and symptom measures
复制标题
DOI：
10.31234/osf.io/rynhk
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
S. Zorowitz;Y. Niv;Daniel Bennett
中科院分区：
文献类型：
--
作者：
S. Zorowitz;Y. Niv;Daniel Bennett
计算精神病学领域的一种常见研究设计涉及利用在线参与者招募的力量来评估认知任务中的行为与大量不同样本中自我报告的精神症状严重程度之间的相关性。尽管大量在线样本对于精神病学研究有许多优势，但这种研究设计的一些潜在陷阱尚未得到广泛了解。在这里，我们详细介绍了由于对精神症状调查中粗心或不努力的反应筛查不充分而导致任务行为和症状严重程度之间可能出现完全虚假相关的情况。具体来说，由于许多精神症状调查在普通人群中的真实得分分布不对称，因此对这些调查不小心回应的参与者将表现出明显升高的症状水平。如果这些参与者在任务表现上同样粗心，并且不被排除在分析之外，则可能会导致较高的症状评分与较差的行为任务表现之间的虚假关联。在这里，我们在 N = 386 名在线招募的参与者中准确展示了这种结果模式，这些参与者完成了自我报告症状组和简短的逆转学习选择任务。我们表明，当被标记为对调查做出粗心回应的参与者被排除在分析之外时，许多行为与症状的相关性就完全消失了。我们还表明，仅基于任务表现的排除不足以防止这些虚假相关性。值得注意的是，我们证明这些虚假相关性的假阳性率随着样本量的增加而增加，这与常见的假设相反。我们为研究人员如何使用这种通用实验设计在未来的研究中防范这一问题提供指导；我们特别建议采用目前在该领域不常见的自我报告措施筛选方法。
A common research design in the field of computational psychiatry involves leveraging the power of online participant recruitment to assess correlations between behavior in cognitive tasks and the self-reported severity of psychiatric symptoms in large, diverse samples. Although large online samples have many advantages for psychiatric research, some potential pitfalls of this research design are not widely understood. Here we detail circumstances in which entirely spurious correlations may arise between task behavior and symptom severity as a result of inadequate screening of careless or low-effort responding on psychiatric symptom surveys. Specifically, since many psychiatric symptom surveys have asymmetric ground-truth score distributions in the general population, participants who respond carelessly on these surveys will show apparently elevated symptom levels. If these participants are similarly careless in their task performance, and are not excluded from analysis, this may result in a spurious association between greater symptom scores and worse behavioral task performance. Here, we demonstrate exactly this pattern of results in N = 386 participants recruited online to complete a self-report symptom battery and a short reversal-learning choice task. We show that many behavior-symptom correlations are entirely abolished when participants flagged for careless responding on surveys are excluded from analysis. We also show that exclusion based on task performance alone is not sufficient to prevent these spurious correlations. Of note, we demonstrate that false-positive rates for these spurious correlations increase with sample size, contrary to common assumptions. We offer guidance on how researchers using this general experimental design can guard against this issue in future research; in particular, we recommend the adoption of screening methods for self-report measures that are currently uncommon in this field.