Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators.
Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators.
复制标题
DOI：
10.1371/journal.pmed.1003368
复制
发表时间：
2020-11
期刊：
影响因子：
15.8
通讯作者：
Egan M
中科院分区：
文献类型：
--
作者：
McGill E;Marks D;Er V;Penney T;Petticrew M;Egan M
公共卫生评价方法因过于简化而受到批评，未能为公共卫生决策提供适当的证据。“复杂系统方法”一直被提倡用来解释现实世界的复杂性。定性方法可能非常适合于理解复杂社会环境中的变化，但是应用复杂系统方法为定性研究提供信息的指导仍然有限和不发达。本系统综述旨在分析利用涉及复杂系统视角的定性方法的已发表的过程评估实例，并提出定性复杂系统过程评估的框架。我们通过检索2014年1月1日至2019年9月30日的电子数据库（Scopus、MEDLINE、Web of Science）、引文检索和专家咨询，对涉及定性方法的复杂系统过程评价进行了系统检索。如果过程评价自我认定采用系统或复杂性导向的方法、综合定性方法、报告的实证结果和评估的公共卫生干预措施，则纳入过程评价。两名审稿人独立评估了每项研究，以确定与系统思维和复杂性科学传统相关的概念。确定了21项独特的研究，评估了城市规划、性健康、预防暴力、药物使用和社区改造等广泛的公共卫生干预措施。评估在13个不同国家（9个高收入国家和4个中等收入国家）的学校、工作场所和社区等环境中进行。所有报告在定性数据分析中使用了一些复杂系统概念。在14项评估中，对复杂系统的考虑影响了干预设计、评估计划或实地工作。已确定的研究使用系统概念来描述和描述某个时间点的系统。只有4个评估明确地利用了一系列复杂性概念来评估系统内的变化，这些变化是随着时间的推移由干预实施引起的，或者与干预实施共同发生的。我们方法的局限性是只包括英语论文，依赖研究作者报告他们对复杂系统概念的使用，以及审稿人对每个研究中涉及的概念的主观判断。这项研究发现，在实践中如何用定性方法将复杂系统的观点引入公共卫生过程评估，并没有达成共识，而且这种性质的许多研究都是在单个时间点描述静态系统。我们建议未来的研究使用两阶段框架进行定性过程评估，从复杂系统的角度评估随时间的变化。第一阶段包括对系统进行描述，并确定系统如何响应干预而发生变化的假设。第二阶段涉及在适应性评估方法中跟踪突发发现的途径。Elizabeth McGill及其同事提出了一个评估公共卫生干预如何与不断变化的系统相互作用的框架。过程评价用于公共卫生，以了解干预措施如何以及为什么起作用（或不起作用），针对哪些人群以及在哪些环境中起作用。过程评估通常使用定性的方法——比如采访人们，观察人们的日常和工作习惯——来得出结论。公共卫生领域的研究人员认为，我们需要以一种考虑到政策和干预措施发生的更广泛系统的方式进行研究——我们称之为“复杂系统视角”。到目前为止，据我们所知，还没有具体的框架来描述研究人员在使用定性方法进行过程评估时如何使用复杂系统的观点。我们进行了系统的文献综述，寻找定性过程评估的例子，这些评估自我认同使用复杂系统的角度来评估公共卫生干预措施。我们发现了21种不同类型的公共卫生干预措施的不同评估，包括针对学生和员工健康、性健康、儿童发展和安全、社区赋权、暴力预防和物质使用的干预措施。我们发现，这些评估描述了公共卫生工作发生的系统，但在分析影响健康的变化如何在这些系统内发生方面效果较差。很少有证据表明，对于如何最好地将复杂系统的观点引入使用定性方法进行评估的过程，特别是如何评估干预措施如何与不断变化的系统相互作用，人们有着共同的理解。我们开发了一个两阶段框架来指导想要应用复杂系统视角进行定性过程评估的研究人员。本综述排除了那些不自认为使用复杂系统视角的研究，因此我们可能错过了使用该视角但未使用相关术语的文献。
Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A “complex systems approach” has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations. We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014–September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study. This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach. Elizabeth McGill and colleagues present a framework for assessing how public health interventions interact with changing systems. Process evaluations are used in public health to understand how and why an intervention works (or does not work), for which population groups, and in which settings. Process evaluations often use qualitative methods—such as interviewing people and observing people in their daily and work routines—in order to draw their conclusions. Researchers in public health have contended that we need to do research in a manner that considers the broader system in which policies and interventions take place—something we call a “complex systems perspective.” To date and to our knowledge, there is no specific framework that describes how researchers can use a complex systems perspective when they conduct a process evaluation with qualitative methods. We conducted a systematic literature review that looked for examples of qualitative process evaluations that self-identify as using a complex systems perspective to evaluate public health interventions. We found 21 different evaluations of many different types of public health interventions, including interventions to address student and employee health, sexual health, child development and safety, community empowerment, violence prevention, and substance use. We found that these evaluations describe the systems in which public health efforts take place but are less effective at analyzing how changes affecting health occur within these systems. There is little evidence of a commonly shared understanding of how best to bring a complex systems perspective to process evaluations using qualitative methods, particularly, how to assess how interventions interact with a changing system. We developed a 2-phase framework to guide researchers who want to apply a complex systems perspective to qualitative process evaluations. This review excluded studies that do not self-identify as using a complex systems perspective so we may have missed literature that uses this perspective but not the associated terminology.
登录
查看更多内容
影响因子：
2.4
作者：
Byrne, David
通讯作者：
Byrne, David
影响因子：
4.5
作者：
Crane, Melanie;Bohn-Goldbaum, Erika;Grunseit, Anne
通讯作者：
Grunseit, Anne
影响因子：
4.8
作者：
Alfandari, Ravit
通讯作者：
Alfandari, Ravit
影响因子：
3.3
作者：
Alfandari, Ravit
通讯作者：
Alfandari, Ravit
影响因子：
4
作者：
Dhanasekar, R.;Jonesh, S.
通讯作者：
Jonesh, S.