The practice of 'doing' evaluation: lessons learned from nine complex intervention trials in action
The practice of 'doing' evaluation: lessons learned from nine complex intervention trials in action
复制标题
DOI：
10.1186/1748-5908-9-75
复制
发表时间：
2014-06-17
影响因子：
7.2
通讯作者：
Chandler, Clare I. R.
中科院分区：
文献类型：
--
作者：
Reynolds, Joanna;DiLiberto, Deborah;Chandler, Clare I. R.
背景：试验人员越来越认识到，在理解特定的“现实生活”环境如何影响复杂卫生干预措施的提供和接受方面存在挑战。对在卫生服务环境中改变卫生工作者和/或患者行为的干预措施的评估就是这些挑战的例证。在解释评估数据时，通过对保真度、覆盖面和强度的过程评估来解释预期干预实施的偏差。但是，没有提出这种系统的方法来解释评价活动在实际中可能偏离解释数据时所作假设的方式。方法：通过集体案例研究，探讨在现实环境中开展评估活动的经验，这些评估活动旨在改善不同卫生服务环境中疟疾的适当诊断和治疗。使用了多种数据来源，包括与调查人员的深度访谈，参与者观察研究，以及几轮讨论和反思。结果和讨论：根据我们进行这些评估的实际经验，我们确定了六个关键的“教训”，关于如何意识到和管理试验结构的各个方面，包括研究人员、实地工作者、参与者和数据收集工具的界面，这些方面可能会影响数据的预期产出和结果的解释。这些课程包括：在整个学习团队中培养对个别实践如何有助于学习目标的共同理解；促进和促进团队内部沟通，以持续反思评估的进展；建立子研究小组之间持续合作和对话的流程；实地研究协调员将日常项目管理与科学监督联系起来的重要性；收集和审查关于评价进展的反思性实地记录，以帮助解释结果；这些方法应该有助于识别和反思评估和干预之间可能存在的重叠。结论：我们得出的经验教训指向了反身性原则，我们认为，反身性原则需要成为评估复杂干预措施的标准实践的一部分，以促进对干预措施效果的更有意义的解释，并更好地为未来的实施和决策提供信息。
Background: There is increasing recognition among trialists of the challenges in understanding how particular 'real-life' contexts influence the delivery and receipt of complex health interventions. Evaluations of interventions to change health worker and/or patient behaviours in health service settings exemplify these challenges. When interpreting evaluation data, deviation from intended intervention implementation is accounted for through process evaluations of fidelity, reach, and intensity. However, no such systematic approach has been proposed to account for the way evaluation activities may deviate in practice from assumptions made when data are interpreted.Methods: A collective case study was conducted to explore experiences of undertaking evaluation activities in the real-life contexts of nine complex intervention trials seeking to improve appropriate diagnosis and treatment of malaria in varied health service settings. Multiple sources of data were used, including in-depth interviews with investigators, participant-observation of studies, and rounds of discussion and reflection.Results and discussion: From our experiences of the realities of conducting these evaluations, we identified six key 'lessons learned' about ways to become aware of and manage aspects of the fabric of trials involving the interface of researchers, fieldworkers, participants and data collection tools that may affect the intended production of data and interpretation of findings. These lessons included: foster a shared understanding across the study team of how individual practices contribute to the study goals; promote and facilitate within-team communications for ongoing reflection on the progress of the evaluation; establish processes for ongoing collaboration and dialogue between sub-study teams; the importance of a field research coordinator bridging everyday project management with scientific oversight; collect and review reflective field notes on the progress of the evaluation to aid interpretation of outcomes; and these approaches should help the identification of and reflection on possible overlaps between the evaluation and intervention.Conclusion: The lessons we have drawn point to the principle of reflexivity that, we argue, needs to become part of standard practice in the conduct of evaluations of complex interventions to promote more meaningful interpretations of the effects of an intervention and to better inform future implementation and decision-making.