Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials.
Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials.
复制标题
DOI：
10.1371/journal.pone.0096920
复制
发表时间：
2014
期刊：
影响因子：
3.7
通讯作者：
Cummings GG
中科院分区：
文献类型：
--
作者：
Armijo-Olivo S;Ospina M;da Costa BR;Egger M;Saltaji H;Fuentes J;Ha C;Cummings GG
通过比较科克伦综述作者与设盲外部综述者的评分，检验RoB工具应用于物理治疗（PT）试验的评分者间可靠性。通过检索科克伦系统评价数据库中PT干预措施的荟萃分析，确定PT中的随机对照试验（RCT）。RoB评估由2名对科克伦综述中报告的RoB评级不知情的评价者独立进行。从科克伦综述中提取RoB评估数据以及综述和试验的其他特征。然后将两名评审员之间的共识评估与来自科克伦评审的RoB评级进行比较。使用加权kappa（κ）评估科克伦和设盲外部评价者之间的一致性。总共评估了17篇科克伦综述中的109项试验。科克伦综述作者和设盲外部综述者之间总体RoB评估的评分者间可靠性较差（κ = 0.02，95%CI：-0.06，0.06]）。 RoB工具各个领域的评估者间可靠性较差（中位数κ = 0.19），范围为κ =-0.04（“其他偏倚”）至κ = 0.62（“序列生成”）。 在荟萃分析水平上，总体RoB评估也没有一致性（κ =−0.29，95%CI：−0.81，0.35]）。 使用RoB工具对RCT进行的偏倚风险评估在不同的研究组中并不一致。不仅在试验水平上，而且在荟萃分析水平上也表现出一致性差。结果对决策有影响，因为根据分析证据的小组，可以得出不同的建议。需要改进指导方针，以一致地应用RoB工具，并对不同卫生领域的工具进行修订。
To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ = 0.02, 95%CI: −0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ = 0.19), ranging from κ = −0.04 (“Other bias”) to κ = 0.62 (“Sequence generation”). There was also no agreement (κ = −0.29, 95%CI: −0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.
登录
查看更多内容
影响因子：
3
作者：
Furlan, Andrea D.;Pennick, Victoria;van Tulder, Maurits
通讯作者：
van Tulder, Maurits
DOI：
10.1136/bmj.b4012
发表时间：
2009-10-19
期刊：
BMJ (Clinical research ed.)
影响因子：
--
作者：
Hartling L;Ospina M;Liang Y;Dryden DM;Hooton N;Krebs Seida J;Klassen TP
通讯作者：
Klassen TP
影响因子：
22.4
作者：
COHEN, J
通讯作者：
COHEN, J
影响因子：
3.7
作者：
Hartling L;Bond K;Vandermeer B;Seida J;Dryden DM;Rowe BH
通讯作者：
Rowe BH
DOI：
10.1016/0197-2456(90)90175-2
发表时间：
1990-10-01
期刊：
CONTROLLED CLINICAL TRIALS
影响因子：
--
作者：
EMERSON, JD;BURDICK, E;CHALMERS, TC
通讯作者：
CHALMERS, TC