A systematic review of measures of implementation players and processes: Summarizing the dearth of psychometric evidence.
A systematic review of measures of implementation players and processes: Summarizing the dearth of psychometric evidence.
复制标题
DOI：
10.1177/26334895211002474
复制
发表时间：
2021-01
影响因子：
--
通讯作者：
Lewis, Cara C.
中科院分区：
文献类型：
--
作者：
Dorsey, Caitlin N.;Mettert, Kayne D.;Puspitasari, Ajeng J.;Damschroder, Laura J.;Lewis, Cara C.
关键词：
测量是任何领域的关键组成部分。系统审查是一种定位度量和发现当前度量实践中的差距的方法。本研究确定了行为健康环境中使用的测量方法，这些测量方法评估了过程域内的所有构念和实施研究统一框架（CFIR）定义的内部设置域的两个构念。虽然以前的概念性工作已经确定了社会网络和关键利益相关者在整个实施过程中的重要性，但测量研究并未侧重于调查这些活动的执行质量。审查分三个阶段进行：第一阶段，数据收集包括(1)搜索字符串生成，(2)标题和摘要筛选，(3)全文审查，(4)映射到cfr结构，(5)“引用”搜索。第二阶段，数据提取，包括与心理测量和语用评定量表（论文）中包含的九个心理测量属性相关的编码信息。第三期完成数据分析。测量方法仅在七个结构中确定：结构特征（n = 13），网络和沟通（n = 29），参与（n = 1），意见领袖（n = 5），冠军（n = 5），规划（n = 5）以及反思和评估（n = 5）。没有确定正式任命的实施领导、外部变更代理人或执行的定量评估措施。内部一致性和规范是最常见的报告，而没有研究报告的区别效度或反应性。样本中没有一个测量报告了论文评估的所有九种心理测量特性。在确定的测量样本中，得分从“-2”到“10”不等，总分为36分。总体测量显示证据不足，可用的心理测量信息有限。大多数是针对特定研究的，限制了它们的普遍性。未来的工作应侧重于更严格的度量开发和当前现有度量的测试，同时远离创建新的、单一用途的度量。我们如何衡量实施循证干预措施所涉及的过程和参与者，对于了解哪些因素有助于或损害干预措施在实践中的使用以及如何扩大干预措施的规模至关重要。不幸的是，对这些因素的衡量——利益相关者、他们的网络和沟通，以及他们的实施活动——很少受到关注。本研究试图确定和评估这些类型的措施的质量。我们的审查侧重于收集用于确定有影响力的工作人员（即意见领袖和拥护者）的措施，并调查他们如何规划、执行、参与和评估艰苦的实施工作。在确定这些测量方法后，我们收集了所有关于其使用的公开信息，以评估其证据的质量，包括它们在每个使用中产生跨项目一致结果的能力（即可靠），以及它们是否评估了它们打算测量的内容（即有效）。我们的搜索找到了40多种用于评估的行为健康设置措施。我们观察到缺乏可靠性和有效性的证据，即使有证据，质量也很低。这些发现告诉我们，需要更多的测量工作来更好地理解如何优化玩家和过程，以实现成功的执行。
Measurement is a critical component for any field. Systematic reviews are a way to locate measures and uncover gaps in current measurement practices. The present study identified measures used in behavioral health settings that assessed all constructs within the Process domain and two constructs from the Inner setting domain as defined by the Consolidated Framework for Implementation Research (CFIR). While previous conceptual work has established the importance social networks and key stakeholders play throughout the implementation process, measurement studies have not focused on investigating the quality of how these activities are being carried out. The review occurred in three phases: Phase I, data collection included (1) search string generation, (2) title and abstract screening, (3) full text review, (4) mapping to CFIR-constructs, and (5) “cited-by” searches. Phase II, data extraction, consisted of coding information relevant to the nine psychometric properties included in the Psychometric And Pragmatic Rating Scale (PAPERS). In Phase III, data analysis was completed. Measures were identified in only seven constructs: Structural characteristics (n = 13), Networks and communication (n = 29), Engaging (n = 1), Opinion leaders (n = 5), Champions (n = 5), Planning (n = 5), and Reflecting and evaluating (n = 5). No quantitative assessment measures of Formally appointed implementation leaders, External change agents, or Executing were identified. Internal consistency and norms were reported on most often, whereas no studies reported on discriminant validity or responsiveness. Not one measure in the sample reported all nine psychometric properties evaluated by the PAPERS. Scores in the identified sample of measures ranged from “-2” to “10” out of a total of “36.” Overall measures demonstrated minimal to adequate evidence and available psychometric information was limited. The majority were study specific, limiting their generalizability. Future work should focus on more rigorous measure development and testing of currently existing measures, while moving away from creating new, single use measures. How we measure the processes and players involved for implementing evidence-based interventions is crucial to understanding what factors are helping or hurting the intervention’s use in practice and how to take the intervention to scale. Unfortunately, measures of these factors—stakeholders, their networks and communication, and their implementation activities—have received little attention. This study sought to identify and evaluate the quality of these types of measures. Our review focused on collecting measures used for identifying influential staff members, known as opinion leaders and champions, and investigating how they plan, execute, engage, and evaluate the hard work of implementation. Upon identifying these measures, we collected all published information about their uses to evaluate the quality of their evidence with respect to their ability to produce consistent results across items within each use (i.e., reliable) and if they assess what they are intending to measure (i.e., valid). Our searches located over 40 measures deployed in behavioral health settings for evaluation. We observed a dearth of evidence for reliability and validity and when evidence existed the quality was low. These findings tell us that more measurement work is needed to better understand how to optimize players and processes for the purposes of successful implementation.
登录
查看更多内容
影响因子：
3.6
作者：
Aarons, Gregory A.;Hoagwood, Kimberly;Landsverk, John;Glisson, Charles;Kelleher, Kelly;Cafri, Guy
通讯作者：
Cafri, Guy
DOI：
10.1186/1748-5908-4-50
发表时间：
2009-08-07
期刊：
Implementation science : IS
影响因子：
--
作者：
Damschroder LJ;Aron DC;Keith RE;Kirsh SR;Alexander JA;Lowery JC
通讯作者：
Lowery JC
DOI：
10.1007/s10488-009-0226-y
发表时间：
2009-09-01
影响因子：
2.6
作者：
Bond, Gary R.;Drake, Robert E.;Xie, Haiyi
通讯作者：
Xie, Haiyi
影响因子：
7.2
作者：
Cook, Joan M.;O'Donnell, Casey;Schnurr, Paula P.
通讯作者：
Schnurr, Paula P.
DOI：
10.1023/b:mhsr.0000024351.12294.65
发表时间：
2004-06-01
期刊：
Mental health services research
影响因子：
--
作者：
Aarons, Gregory A
通讯作者：
Aarons, Gregory A