Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention.
Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention.
复制标题
DOI：
10.1007/s10488-014-0538-4
复制
发表时间：
2015-09
影响因子：
2.6
通讯作者：
Brown, C. Hendricks
中科院分区：
文献类型：
--
作者：
Gallo, Carlos;Pantin, Hilda;Villamar, Juan;Prado, Guillermo;Tapia, Maria;Ogihara, Mitsunori;Cruden, Gracelyn;Brown, C. Hendricks
关键词：
认真的忠诚度监测和反馈对于实施有效的干预措施至关重要。存在各种各样的程序来评估保真度;大多数来自观察评估。然而，这些保真度的措施是资源密集型的研究团队在疗效/有效性试验，往往是无法实现或无法管理的主办组织的评级时，该计划是在大规模实施。我们提出了第一步的语言模式的自动化处理的保真度监测的行为干预，使用创新的混合方法的保真度评估方法，使用基于规则的，计算语言学，以克服主要的资源负担。数据来自Familias Unidas干预措施的有效性试验，这是一种以证据为基础，以家庭为中心的预防性干预措施，发现在减少西班牙裔青年的行为问题，药物使用和艾滋病毒性风险行为方面有效。这种计算方法侧重于“加入”，衡量协调员与家庭的工作联盟的质量。提供了可靠性的定量评估。对于衡量连接的新方法，人类评分员和机器评分员之间的Kappa得分达到了0.83。早期的研究结果表明，这种方法可以降低忠诚度测量的高成本和忠诚度评估和反馈到促进者之间的时间延迟;它也有可能提高干预忠诚度评级的质量。
Careful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments. However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on “joining,” which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached .83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings.
登录
查看更多内容
DOI：
10.1007/s10488-011-0387-3
发表时间：
2012-07-01
影响因子：
2.6
作者：
Brown, C. Hendricks;Kellam, Sheppard G.;McManus, John W.
通讯作者：
McManus, John W.
影响因子：
3.9
作者：
Hogue, Aaron;Dauber, Sarah;Liddle, Howard A.
通讯作者：
Liddle, Howard A.
影响因子：
3.1
作者：
Chambers, David A.
通讯作者：
Chambers, David A.
DOI：
10.1207/s15374424jccp3102_02
发表时间：
2002-06-01
影响因子：
4.2
作者：
Henggeler, SW;Schoenwald, SK;Edwards, DL
通讯作者：
Edwards, DL
影响因子：
14.4
作者：
de Cooman, G;Zaffalon, M
通讯作者：
Zaffalon, M