Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions.
Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions.
复制标题
DOI：
10.1186/s12874-017-0325-5
复制
发表时间：
2017-03-23
影响因子：
4
通讯作者：
Hartling L
中科院分区：
文献类型：
--
作者：
Pollock M;Fernandes RM;Hartling L
综述（overviews）汇编来自多项系统性综述的信息，为决策提供单一的相关证据综合。建议作者在综述中评估和报告SR的方法学质量-例如，使用评估系统综述的测量工具（AMSTAR）。目前，在综述作者是否以及如何评估和报告SR质量方面存在差异，并且可用的指导有限。我们的目标是：检查在医疗保健干预措施综述中使用AMSTAR评估科克伦和非科克伦SR质量时涉及的方法学考虑;识别在综述中使用AMSTAR时的挑战（并制定潜在的决策规则）;检查在综述中做出纳入决策时考虑方法学质量的潜在影响。我们选择了7个医疗保健干预措施的概述，并包括所有符合每个概述的纳入标准的SR。对于每个SR，两名评审员独立进行了AMSTAR评估，并达成共识，并讨论了遇到的挑战。我们还研究了AMSTAR评估和SR结果/结论之间的相关性。纳入了95例SR（30例科克伦，65例非科克伦）。与非科克伦SR相比，科克伦SR的平均AMSTAR评估（9.6/11 vs. 5.5/11; p < 0.001）和评分者间可靠性（AC 1统计量：0.84 vs. 0.69;使用Landis & Koch标准的“几乎完美”vs.“实质性”）更高。在综述中应用AMSTAR时，确定了四个挑战：员工代表和综述的范围往往不同;研究类似主题的员工代表有时会做出不同的方法决定;非Cochrane员工代表的报告有时很差;一些非Cochrane员工代表包括其他员工代表以及主要研究。制定决策规则以应对每一项挑战。我们没有发现AMSTAR评估与SR结果/结论相关的证据。结果表明，AMSTAR工具可以成功地用于概述，包括科克伦和非科克伦SR，虽然决策规则可能是有用的，以规避常见的挑战。调查结果支持现有的建议，即对概述中的工作人员代表的质量评估应独立进行，一式两份，并有一个协商一致的进程。结果还表明，使用方法学质量来指导纳入决策（例如，排除开展和报告不佳的工作人员代表）可能不会在概述过程中引入偏倚。本文的在线版本（doi：10.1186/s12874-017-0325-5）包含补充材料，可供授权用户使用。
Overviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews—for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews. We selected seven overviews of healthcare interventions and included all SRs meeting each overview’s inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions. Ninety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; “almost perfect” vs. “substantial” using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions. Results indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process. The online version of this article (doi:10.1186/s12874-017-0325-5) contains supplementary material, which is available to authorized users.