Psychometric assessment of three newly developed implementation outcome measures
Psychometric assessment of three newly developed implementation outcome measures
复制标题
DOI：
10.1186/s13012-017-0635-3
复制
发表时间：
2017-08-29
影响因子：
7.2
通讯作者：
Halko, Heather
中科院分区：
文献类型：
--
作者：
Weiner, Bryan J.;Lewis, Cara C.;Halko, Heather
背景：执行成果措施对于监测和评价执行工作的成功至关重要。然而，目前可用的措施缺乏概念上的明确性，并在很大程度上未知的可靠性和有效性。本研究开发和心理测量评估三个新的措施：干预措施的可接受性（AIM），干预适当性措施（IAM），和干预措施的可行性（EQUIPMENT）。方法：36个实施科学家和27名心理健康专业人员分配31个项目的结构，并在他们的任务的信心。使用Wilcoxon单样本符号秩和检验评估实质和判别内容效度。探索性和验证性因素分析（EFA和CFA）和Cronbach α被用来评估的概念模型的效度。326名心理健康咨询师阅读了随机分配的六个小插曲之一，描述了一位治疗师正在考虑采用循证实践（EBP）。参与者使用15个项目来评估治疗师对采用EBP的可接受性、适当性和可行性的看法。CFA和Cronbach α被用来完善量表，评估结构效度，并评估信度。方差分析（ANOVA）被用来评估已知组的有效性。最后，一半的辅导员被随机分配到接收相同的小插曲和另一半相反的小插曲，并要求所有重新评级的可接受性，适当性和可行性。Pearson相关系数被用来评估重测信度和线性回归来评估变化的敏感性。每个结构5个项目的修剪CFA表现出可接受的模型拟合（CFI = 0.98，RMSEA = 0.08）和高因子载荷（0.79至0.94）。5项量表的α值在0.87和0.89之间。尺度细化的基础上测量特定的CFA和克朗巴赫阿尔法使用小插曲数据产生4项量表（a的从0.85到0.91）。一个三因素的CFA表现出可接受的拟合（CFI = 0.96，RMSEA = 0.08）和高因子负荷（0.75至0.89），表明结构有效性。方差分析显示显着的主效应，表明已知组的有效性。重测信度系数为0.73 ~ 0.88。回归分析表明，每项措施是敏感的变化在两个directions.Conclusions：目的，IAM，并证明有前途的心理测量特性。计划进行预测有效性评估。
Background: Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).Methods: Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change.Results: All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (a's from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions.Conclusions: The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.