PsyRAT: Extensible Open-Source Software for Applying Generalizability Theory to Assess Psychometric Reliability of Trial-Wise Scores and Optimize Tasks for RDoC

PsyRAT：可扩展的开源软件，用于应用概括性理论来评估试验分数的心理测量可靠性并优化 RDoC 任务

• 批准号: 10676972
• 负责人: Peter Eugene Clayson
• 金额: $ 57.62万
• 依托单位: UNIVERSITY OF SOUTH FLORIDA
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-04 至 2027-06-30
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10676972
• 关键词: Adoption; Anxiety; Attention; Behavior; Biological; Biological Markers; Brain; Characteristics; Clinical; Clinical Research; Clinical Trials; Code; Computer software; Data; Decision Making; Development; Diagnosis; Diagnostic; Dimensions; E-learning; Educational Materials; Electroencephalography; Electromyography; Evaluation; Evaluation Studies; Event; Event-Related Potentials; Face; Fostering; Frequencies; Functional Magnetic Resonance Imaging; Funding; Genomics; Goals; Guidelines; Individual; Individual Differences; Length; Link; Measurement; Measures; Mental Depression; Mental disorders; Monitor; National Institute of Mental Health; Neurosciences; Outcome; Participant; Patient Self-Report; Persons; Procedures; Process; Property; Psychometrics; Psychopathology; Psychophysiology; Public Health; Reflex action; Reporting; Research; Research Domain Criteria; Research Personnel; Research Project Grants; Resources; Rewards; Risk; Risk Estimate; Source; Symptoms; System; Testing; Time; Training; Work; behavior measurement; biomarker evaluation; biomarker performance; biomarker selection; candidate marker; clinical biomarkers; data access; design; experience; improved; innovation; neglect; neural; novel; novel strategies; online tutorial; open source; open source tool; precision medicine; process optimization; psychologic; response; software development; theories; tool; tool development; user-friendly

PROJECT SUMMARY/ABSTRACT Neural biomarkers are an important focus of the NIMH Research Domain Criteria (RDoC) initiative, and they are increasingly used in the context of genomic studies and clinical trials. The use of biomarkers with strong psychometric reliability increases the likelihood of finding replicable effects, improves the validity of their interpretation, and decreases the likelihood of missing real phenomena. Although fundamental psychometric principles have long been a prominent concern among studies that use self-report measures, these principles are underappreciated in studies of psychopathology that use biological measures. This lack of attention to reliability limits the more widespread application of biomarkers in psychopathology and likely contributes to replication problems. Generalizability theory is a multifaceted framework for identifying sources of measurement error, and this framework is uniquely suited to assessing the reliability of biological measures and to optimizing tasks for reliability. A critical need exists for tractable software to facilitate the application of generalizability theory to time-frequency electroencephalography (EEG), event-related potentials (ERPs), facial electromyography (EMG), and electrodermal activity (EDA). The objective of this project in response to PAR- 18-930 on measurement tool development for RDoC is to (i) develop an extensive treatment of generalizability theory for psychopathology researchers, (ii) develop accessible software to implement it, (iii) show how to apply these resources to optimize paradigms for individual-differences research, and (iv) disseminate the software with a user-friendly guide. This project will facilitate the routine evaluation of reliability through these specific aims: 1) Design and implement generalizability theory formulas for evaluating group- and subject-level reliability for paradigm optimization; 2) Develop software to implement these formulas with data from widely used psychophysiological software; 3) Apply results to optimize three commonly studied tasks; and 4) Develop online educational material on the application of these resources to novel paradigms and measures. This research project is innovative, because it represents a substantive departure from standard practice by shifting the focus to the reliability of data from individuals, rather than groups, to identify sources of measurement error and minimize their impact. This work promotes best practices in reporting psychometric properties of biological measures and is applicable to data from any task with trial-wise scores. The resulting open-source toolbox, the Psychophysiologist’s Reliability Analysis Toolbox (PsyRAT), can facilitate guidelines for optimizing paradigms, making decisions about individual-subject data, and grounding individual-differences questions (central to clinical research, especially for applications in precision medicine) in measures of reliability. The proposed process for guiding biomarker evaluation through high-quality psychometrics will pave the way for better selection of biomarkers and task development, ultimately improving the clinical utility of these biomarkers.

项目总结/摘要 神经生物标志物是NIMH研究领域标准（RDoC）倡议的一个重要焦点， 越来越多地用于基因组研究和临床试验。使用生物标记物， 心理测量可靠性增加了发现可复制效应的可能性，提高了其有效性。 解释，并降低了错过真实的现象的可能性。虽然基本的心理测量 原则一直是使用自我报告措施的研究中的一个突出问题，这些原则 在使用生物测量的精神病理学研究中被低估了。这种缺乏关注， 可靠性限制了生物标志物在精神病理学中更广泛的应用，并可能有助于 复制问题。概化理论是一个多方面的框架，用于确定 测量误差，这个框架是唯一适合于评估生物措施的可靠性 以及优化任务以提高可靠性。迫切需要易于处理的软件，以促进 时频脑电图（EEG）、事件相关电位（ERP）、面部表情、 肌电图（EMG）和皮肤电活动（EDA）。该项目的目标是响应PAR- 18-930关于RDoC测量工具开发的主要目的是（i）开发一种广泛的通用性处理方法， 理论的精神病理学研究人员，（ii）开发可访问的软件来实现它，（iii）显示如何 应用这些资源，优化个体差异研究的范式，以及（iv）传播 具有用户友好指南的软件。本项目将通过以下方式促进可靠性的常规评估： 具体目标：1）设计并实现用于评估群体和学科水平的概化理论公式 2）开发软件，利用广泛的数据实现这些公式， 使用心理生理学软件; 3）应用结果优化三个常用的研究任务;和4）开发 关于将这些资源应用于新的范例和措施的在线教育材料。这 研究项目是创新的，因为它代表了一个实质性的偏离标准做法， 关注来自个人而不是群体的数据的可靠性，以确定测量误差的来源 并尽量减少其影响。这项工作促进了在报告生物学的心理测量特性方面的最佳做法。 测量并适用于任何具有试验分数的任务的数据。由此产生的开源工具箱， 心理生理学家的可靠性分析模型（PsyRAT），可以促进优化范式的指导方针， 对个体受试者数据做出决定，并将个体差异问题作为基础（ 临床研究，特别是在精密医学中的应用）的可靠性措施。拟议 通过高质量的心理测量学指导生物标志物评估的过程将为更好地 生物标志物的选择和任务开发，最终提高这些生物标志物的临床效用。