Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records
Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records
复制标题
DOI：
10.1002/sim.1190
复制
发表时间：
2002-06-15
影响因子：
2
通讯作者：
Wentz, R
中科院分区：
文献类型：
--
作者：
Edwards, P;Clarke, M;Wentz, R
进行了一项研究，以评估审查人员在筛选相关试验记录以进行系统审查时的准确性和可靠性。对十个电子书目数据库的敏感搜索产生了 22571 条可能相关试验的记录。记录被分配给四位评审员，以便两名评审员检查每条记录，以便每个评审员对试验的识别可以与其他评审员识别的试验进行比较。使用 Cohen 的 kappa 统计来评估审阅者之间的一致性，使用确定交叉方法来估计审阅者可能错过的试验数量。两名研究人员获得了报告的完整副本并独立评估了审查的资格。合格的报告形成了“黄金标准”，审稿人根据该标准评估筛选的准确性。经过筛选后，22571 条记录中的 301 条被至少一名审阅者识别为潜在相关。两对内的一致性为“几乎完美”（kappa > 0.8），三对内的一致性为“显着”（kappa > 0.6），一对内的一致性为“中等”（kappa > 0.4）。在选定的 301 条记录中，有 273 份完整报告可用。当成对的审阅者就记录的潜在相关性达成一致时，81% 的审阅者符合资格（范围为 69% 至 91%）。如果评审者不同意，则 22% 的人符合资格（范围为 12% 至 45%）。单个审稿人平均漏掉了 8% 的合格报告（范围为 0% 到 24%），而成对的审稿人则没有漏掉任何报告（范围为 0% 到 1%）。使用两名审稿人筛选记录使随机试验的数量平均增加了 9%（范围为 0% 到 32%）。在筛选数千条记录的资格时，审核者可以可靠地识别潜在的相关记录。两名评审员应尽可能筛选记录的资格，以便最大限度地确定相关试验。版权所有 (C) 2002 John Wiley Sons, Ltd.
A Study was conducted to estimate the accuracy and reliability of reviewers when screening records for relevant trials for a systematic review. A sensitive search of ten electronic bibliographic databases yielded 22571 records of potentially relevant trials. Records were allocated to four reviewers such that two reviewers examined each record and so that identification of trials by each reviewer could be compared with those identified by each of the other reviewers. Agreement between reviewers was assessed using Cohen's kappa statistic, Ascertainment intersection methods were used to estimate the likely number of trials missed by reviewers. Full copies of reports were obtained and assessed independently by two researchers for eligibility for the review. Eligible reports formed the 'gold standard' against which an assessment was made about the accuracy of screening by reviewers. After screening, 301 of 22571 records were identified by at least one reviewer as potentially relevant. Agreement was 'almost perfect' (kappa > 0.8) within two pairs, 'substantial' (kappa > 0.6) within three pairs and 'moderate' (kappa > 0.4) within one pair. Of the 301 records selected, 273 complete reports were available. When pairs of reviewers agreed on the potential relevance of records, 81 per cent were eligible (range 69 to 91 per cent). If reviewers disagreed, 22 per cent were eligible (range 12 to 45 per cent). Single reviewers missed on average 8 per cent of eligible reports (range 0 to 24 per cent), whereas pairs of reviewers did not miss any (range 0 to I per cent), The use of two reviewers to screen records increased the number of randomized trials identified by an average of 9 per cent (range 0 to 32 per cent). Reviewers can reliably identify potentially relevant records when screening thousands of records for eligibility. Two reviewers should screen records for eligibility, whenever possible, in order to maximize ascertainment of relevant trials. Copyright (C) 2002 John Wiley Sons, Ltd.