PsyRAT: Extensible Open-Source Software for Applying Generalizability Theory to Assess Psychometric Reliability of Trial-Wise Scores and Optimize Tasks for RDoC

项目摘要/摘要 神经生物标记物是NIMH研究领域标准(RDoC)倡议的一个重要焦点，它们 越来越多地被用于基因组研究和临床试验。生物标志物的使用与强 心理测量学的可靠性增加了发现可复制效应的可能性，提高了他们的有效性 解释，并减少遗漏真实现象的可能性。尽管基本心理测量学 长期以来，在使用自我报告测量的研究中，原则一直是一个突出的问题，这些原则 在使用生物测量的精神病理学研究中被低估了。对此缺乏关注 可靠性限制了生物标记物在精神病理学中更广泛的应用，并可能有助于 复制问题。概括性理论是一个多方面的框架，用于识别 测量误差，该框架特别适合于评估生物测量的可靠性 以及优化任务的可靠性。迫切需要易处理的软件来促进应用 时频脑电(EEG)、事件相关电位(ERPs)、面部的泛化理论 肌电(EMG)和皮肤电活动(EDA)。本项目的目标是响应PAR- 18-930关于RDoC的测量工具开发的目的是(I)开发泛化的广泛处理 精神病理学研究人员的理论，(Ii)开发可访问的软件来实现它，(Iii)展示如何 应用这些资源来优化个体差异研究的范例，以及(Iv)传播 带有用户友好指南的软件。该项目将通过以下方式促进可靠性的常规评估 具体目标：1)设计和实施群体和学科水平评估的概括性理论公式 用于范例优化的可靠性；2)开发软件以使用来自广泛的数据来实施这些公式 使用心理生理学软件；3)应用结果优化三个常用的研究任务；4)开发 关于将这些资源应用于新的范例和措施的在线教育材料。这 研究项目是创新的，因为它代表了对标准做法的实质性偏离 重点关注来自个人而不是群体的数据的可靠性，以确定测量误差的来源 并将其影响降至最低。这项工作促进了在报告生物的心理测量特性方面的最佳实践 测量并适用于具有试用分数的任何任务的数据。由此产生的开源工具箱、 心理生理学家的可靠性分析工具箱，可以促进优化范例的指南， 就个人主题数据做出决策，并确定个人差异问题的基础(核心 临床研究，特别是在精密医学中的应用)。建议数 通过高质量心理测量学指导生物标记物评估的过程将为更好地 生物标记物的选择和任务开发，最终提高这些生物标记物的临床实用性。

PROJECT SUMMARY/ABSTRACT Neural biomarkers are an important focus of the NIMH Research Domain Criteria (RDoC) initiative, and they are increasingly used in the context of genomic studies and clinical trials. The use of biomarkers with strong psychometric reliability increases the likelihood of finding replicable effects, improves the validity of their interpretation, and decreases the likelihood of missing real phenomena. Although fundamental psychometric principles have long been a prominent concern among studies that use self-report measures, these principles are underappreciated in studies of psychopathology that use biological measures. This lack of attention to reliability limits the more widespread application of biomarkers in psychopathology and likely contributes to replication problems. Generalizability theory is a multifaceted framework for identifying sources of measurement error, and this framework is uniquely suited to assessing the reliability of biological measures and to optimizing tasks for reliability. A critical need exists for tractable software to facilitate the application of generalizability theory to time-frequency electroencephalography (EEG), event-related potentials (ERPs), facial electromyography (EMG), and electrodermal activity (EDA). The objective of this project in response to PAR- 18-930 on measurement tool development for RDoC is to (i) develop an extensive treatment of generalizability theory for psychopathology researchers, (ii) develop accessible software to implement it, (iii) show how to apply these resources to optimize paradigms for individual-differences research, and (iv) disseminate the software with a user-friendly guide. This project will facilitate the routine evaluation of reliability through these specific aims: 1) Design and implement generalizability theory formulas for evaluating group- and subject-level reliability for paradigm optimization; 2) Develop software to implement these formulas with data from widely used psychophysiological software; 3) Apply results to optimize three commonly studied tasks; and 4) Develop online educational material on the application of these resources to novel paradigms and measures. This research project is innovative, because it represents a substantive departure from standard practice by shifting the focus to the reliability of data from individuals, rather than groups, to identify sources of measurement error and minimize their impact. This work promotes best practices in reporting psychometric properties of biological measures and is applicable to data from any task with trial-wise scores. The resulting open-source toolbox, the Psychophysiologist’s Reliability Analysis Toolbox (PsyRAT), can facilitate guidelines for optimizing paradigms, making decisions about individual-subject data, and grounding individual-differences questions (central to clinical research, especially for applications in precision medicine) in measures of reliability. The proposed process for guiding biomarker evaluation through high-quality psychometrics will pave the way for better selection of biomarkers and task development, ultimately improving the clinical utility of these biomarkers.