Comparison of Fidelity Assessment Methods

保真度评估方法的比较

• 批准号: 8600181
• 负责人: Angela Leigh Rollins
• 金额: --
• 依托单位: RLR VA MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-11-01 至 2013-10-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8600181
• 关键词: Address; Adherence; Affect; Agreement; Caring; Case Management; Communities; Consensus; Data; Disabled Persons; Effectiveness; Ensure; Equilibrium; Evaluation; Evidence based practice; Future; Gold; Health Planning; Health system; Hospitalization; Human Resources; Individual; Intervention; Measures; Mental Health; Mental Health Services; Mental disorders; Methods; Modeling; Monitor; Outcome; Patient Self-Report; Performance; Persons; Policies; Productivity; Program Evaluation; Protocols documentation; Quality of Care; Recruitment Activity; Relative (related person); Self Assessment; Self-Administered; Services; Site; Site Visit; System; Telephone; Testing; Time; Validation; Veterans; Visit; base; community based service; cost; cost effective; design; evidence base; experience; falls; improved; innovation; meetings; policy implication; programs; psychosocial; severe mental illness; standard care

Background: National policy has dramatically increased the emphasis on implementing evidence-based mental health services to meet the needs of people with severe mental illness, and the VHA has made great strides at providing effective, community-based services. One of the cornerstones of the VHA approach is Mental Health Intensive Case Management (MHICM), a model that is based on one of the most well-defined and empirically supported approaches: assertive community treatment. Most recently, VHA policy shifts have resulted in a proposed set of uniform mental health services to ensure access to a standard set of high quality mental health services, such as MHICM, across the entire VHA. However, successful implementation of evidence-based practices on a broad scale requires psychometrically valid, yet practical, ways to assess and monitor degree of implementation (i.e., fidelity). Currently, the only rigorous method to monitor implementation is an on-site fidelity visit, which is a very time-intensive, expensive, and burdensome approach for both the assessor and the program. Objective: The primary objective of the proposed study is to examine the effectiveness of innovative and potentially cost-effective methods to ensure the quality of mental health services for the most severely disabled veterans with mental illness. Specifically, we will examine the reliability and concurrent validity, predictive validity, and costs associated with three different methods of fidelity assessment: self-report, phone-based remote assessment, and a "gold-standard" on-site fidelity assessment. Methods: Currently all MHICM teams (n=111) provide self-reported fidelity annually to the Northeast Program Evaluation Center (NEPEC). VHA tracks hospitalization outcomes over time. We will recruit 32 teams to participate in a phone-based assessment and an on-site fidelity visit with experienced fidelity assessors. We will stratify teams on type of VHA facility and previous year's performance on the self-assessment. The order of phone and on-site assessments will be counter-balanced, with separate assessors, to reduce potential bias. We will examine level of agreement between fidelity approaches with intraclass-correlations. We will examine predictive validity using regression modeling to examine the association between fidelity method and improved hospitalization outcomes. For our cost identification analysis, we will compare costs across the three methods of assessment, using a mixed (repeated measures) model approach. We will also include a formative evaluation to inform future dissemination of fidelity assessment methods in the VHA and elsewhere.

背景：国家政策大大增加了对实施的重视