Assessing the applicability of public health intervention evaluations from one setting to another: a methodological study of the usability and usefulness of assessment tools and frameworks
Assessing the applicability of public health intervention evaluations from one setting to another: a methodological study of the usability and usefulness of assessment tools and frameworks
复制标题
DOI：
10.1186/s12961-018-0364-3
复制
发表时间：
2018-09-04
影响因子：
4
通讯作者：
Thomas, James
中科院分区：
文献类型：
--
作者：
Burchett, Helen Elizabeth Denise;Blanchard, Laurence;Thomas, James
背景：公共卫生干预措施可以是复杂的，复杂的和背景依赖性，使评估的适用性具有挑战性。然而，为了使它们在原始研究环境之外发挥作用，它们需要可推广到其他环境，至关重要的是，研究用户需要能够确定它可能适用于哪些环境。有许多工具都有评估普遍性/适用性的既定标准，但似乎很少有被广泛使用的工具，而且对于应该使用哪些工具或何时使用也没有共识。这种方法学研究的目的是测试这些工具，以评估他们是多么容易使用，如何有用，他们似乎是。方法：我们确定了现有的审查和更新的搜索工具。根据预先规定的标准筛选参考文献。包括的工具进行了测试，使用它们来评估瑞典体重管理干预的适用性，以英语为背景。研究人员的评估和思考的可用性和实用性的工具收集使用标准的形式。它们的长度、内容、风格和完成所需的时间各不相同。没有一种工具被认为是评估适用性的理想工具。它们的局限性包括不切实际的标准（需要没有资料），注重执行而忽视可转让性（即很少关注新环境下的潜在有效性），标准过于宽泛（与低可靠性相关），以及缺乏对干预措施如何发挥作用的明确关注结论：提出可随时使用的标准的工具可能不是适用性评估的最佳方法。它们可能太长或不完整，过于侧重于差异，未能处理对感兴趣的具体专题至关重要的内容。现在是时候从制定在文献中没有广泛使用的标准清单，到创建一种新的适用性评估方法。关注作用机制，而不是仅仅关注特征，可能是一种有用的方法，但在当前工具中仍未得到充分利用。需要开发、测试、报告和讨论从清单式评估演变而来的评估普遍性的新方法。
Background: Public health interventions can be complicated, complex and context dependent, making the assessment of applicability challenging. Nevertheless, for them to be of use beyond the original study setting, they need to be generalisable to other settings and, crucially, research users need to be able to identify to which contexts it may be applicable. There are many tools with set criteria for assessing generalisability/applicability, yet few seem to be widely used and there is no consensus on which should be used, or when. This methodological study aimed to test these tools to assess how easy they were to use and how useful they appeared to be.Methods: We identified tools from an existing review and an update of its search. References were screened on pre-specified criteria. Included tools were tested by using them to assess the applicability of a Swedish weight management intervention to the English context. Researcher assessments and reflections on the usability and utility of the tools were gathered using a standard pro-forma.Results: Eleven tools were included. Their length, content, style and time required to complete varied. No tool was considered ideal for assessing applicability. Their limitations included unrealistic criteria (requiring unavailable information), a focus on implementation to the neglect of transferability (i.e. little focus on potential effectiveness in the new setting), overly broad criteria (associated with low reliability), and a lack of an explicit focus on how interventions worked (i.e. their mechanisms of action).Conclusion: Tools presenting criteria ready to be used may not be the best method for applicability assessments. They are likely to be either too long or incomplete, too focused on differences and fail to address elements that matter for the specific topic of interest. It is time to progress from developing lists of set criteria that are not widely used in the literature, to creating a new approach to applicability assessment. Focusing on mechanisms of action, rather than solely on characteristics, could be a useful approach, and one that remains underutilised in current tools. New approaches to assessing generalisability that evolve away from checklist style assessments need to be developed, tested, reported and discussed.