Comparison of Fidelity Assessment Methods

保真度评估方法的比较

• 批准号: 8208278
• 负责人: Angela Leigh Rollins
• 金额: --
• 依托单位: RLR VA MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-11-01 至 2013-10-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8208278
• 关键词: Address; Adherence; Affect; Agreement; Caring; Case Management; Communities; Consensus; Data; Disabled Persons; Effectiveness; Ensure; Equilibrium; Evaluation; Evidence based practice; Future; Gold; Health Planning; Health system; Hospitalization; Human Resources; Individual; Intervention; Measures; Mental Health; Mental Health Services; Mental disorders; Methods; Modeling; Monitor; Outcome; Patient Self-Report; Performance; Persons; Policies; Productivity; Program Evaluation; Protocols documentation; Quality of Care; Recruitment Activity; Relative (related person); Self Assessment; Self-Administered; Services; Site; Site Visit; System; Telephone; Testing; Time; Validation; Veterans; Visit; base; community based service; cost; cost effective; design; evidence base; experience; falls; improved; innovation; meetings; policy implication; programs; psychosocial; severe mental illness; standard care

Background: National policy has dramatically increased the emphasis on implementing evidence-based mental health services to meet the needs of people with severe mental illness, and the VHA has made great strides at providing effective, community-based services. One of the cornerstones of the VHA approach is Mental Health Intensive Case Management (MHICM), a model that is based on one of the most well-defined and empirically supported approaches: assertive community treatment. Most recently, VHA policy shifts have resulted in a proposed set of uniform mental health services to ensure access to a standard set of high quality mental health services, such as MHICM, across the entire VHA. However, successful implementation of evidence-based practices on a broad scale requires psychometrically valid, yet practical, ways to assess and monitor degree of implementation (i.e., fidelity). Currently, the only rigorous method to monitor implementation is an on-site fidelity visit, which is a very time-intensive, expensive, and burdensome approach for both the assessor and the program. Objective: The primary objective of the proposed study is to examine the effectiveness of innovative and potentially cost-effective methods to ensure the quality of mental health services for the most severely disabled veterans with mental illness. Specifically, we will examine the reliability and concurrent validity, predictive validity, and costs associated with three different methods of fidelity assessment: self-report, phone-based remote assessment, and a "gold-standard" on-site fidelity assessment. Methods: Currently all MHICM teams (n=111) provide self-reported fidelity annually to the Northeast Program Evaluation Center (NEPEC). VHA tracks hospitalization outcomes over time. We will recruit 32 teams to participate in a phone-based assessment and an on-site fidelity visit with experienced fidelity assessors. We will stratify teams on type of VHA facility and previous year's performance on the self-assessment. The order of phone and on-site assessments will be counter-balanced, with separate assessors, to reduce potential bias. We will examine level of agreement between fidelity approaches with intraclass-correlations. We will examine predictive validity using regression modeling to examine the association between fidelity method and improved hospitalization outcomes. For our cost identification analysis, we will compare costs across the three methods of assessment, using a mixed (repeated measures) model approach. We will also include a formative evaluation to inform future dissemination of fidelity assessment methods in the VHA and elsewhere.

背景：国家政策大幅加大对落实的重视程度 循证精神卫生服务，以满足严重精神疾病患者的需求， VHA 在提供有效的、基于社区的服务方面取得了长足的进步。之一 VHA 方法的基石是心理健康强化病例管理 (MHICM)， 该模型基于最明确定义和经验支持的方法之一： 积极的社区治疗。最近，VHA 政策转变导致了一系列提议 统一的精神卫生服务，以确保获得一套标准的高质量精神卫生服务 整个 VHA 的卫生服务，例如 MHICM。然而，成功实施 大规模的循证实践需要有效且实用的心理测量方法 评估和监控实施程度（即保真度）。目前，唯一严格的 监督实施的方法是现场保真度访问，这是一个非常耗时的过程， 对于评估者和项目来说都是昂贵且繁重的方法。 目的：拟议研究的主要目的是检验有效性 确保心理健康质量的创新且具有潜在成本效益的方法 为患有精神疾病的最严重残疾退伍军人提供服务。具体来说，我们将 检查可靠性和并发有效性、预测有效性和相关成本 三种不同的保真度评估方法：自我报告、基于电话的远程评估、 以及“黄金标准”现场保真度评估。 方法：目前，所有 MHICM 团队 (n=111) 每年都会向参与者提供自我报告的忠诚度 东北项目评估中心（NEPEC）。 VHA 跟踪住院结果超过 时间。我们将招募32个团队参加电话评估和现场保真度 拜访经验丰富的保真度评估员。我们将根据 VHA 设施的类型对团队进行分层 对上一年的表现进行自我评估。电话、现场顺序 评估将由单独的评估员进行平衡，以减少潜在的偏见。我们 将检查保真度方法与类内相关性之间的一致性程度。我们将 使用回归模型检查预测有效性，以检查之间的关联 保真方法并改善住院结果。对于我们的成本识别分析，我们 将比较三种评估方法的成本，使用混合（重复） 措施）模型方法。我们还将进行形成性评估，为未来提供信息 在 VHA 和其他地方传播保真度评估方法。