Does masking author identity improve peer review quality? - A randomized controlled trial
Does masking author identity improve peer review quality? - A randomized controlled trial
复制标题
DOI：
10.1001/jama.280.3.240
复制
发表时间：
1998-07-15
影响因子：
120.7
通讯作者：
Rennie, D
中科院分区：
文献类型：
--
作者：
Justice, AC;Cho, MK;Rennie, D
背景：在审稿人眼中，所有作者可能并不平等。具体来说，知名作者可能会收到不太客观（质量较差）的评论。单一期刊的一项研究发现，当审稿人不知道作者身份时，审稿质量会略有提高。目标。-确定对审稿人不透露作者身份通常是否与生物医学期刊较高的审稿质量相关，并确定常规屏蔽技术的成功。设计和设置。-对提交给《急诊医学年鉴》、《内科医学年鉴》、《美国医学会杂志》、《妇产科》和《医学年鉴》的稿件进行外部审查，进行了一项随机对照试验。眼科干预措施-两名同行审阅了每份手稿。在一个研究组中，两位同行评审员都根据通常的掩盖做法收到了手稿。在另一组中，一名审稿人被随机接收一份作者身份被掩盖的手稿，另一名审稿人收到一份未掩盖的手稿。 主要结果衡量标准：由稿件作者和编辑根据 5 点李克特量表进行评审质量。 0.5 或更大的差异被认为是重要的。结果。-总共 118 份手稿被随机分配，其中 26 份属于常规做法，92 份属于干预。在干预组中，92 篇稿件中的 77 篇（84%）已完成编辑质量评估。 74 篇稿件中有 40 篇（54%）完成了作者质量评估。作者和编辑认为屏蔽（平均差异，0.1；95% 置信区间 [CI]，-0.2 至 0.4）和未屏蔽（平均差异，-0.1；95% CI，-0.5 至 0.4）评论之间的质量没有显着差异。我们还发现评论对编辑决策的影响程度没有差异（平均差异，-0.1；95% CI，-0.3 至 0.3）。掩蔽通常不成功（总体而言，68% 成功掩蔽；95% CI，58%-77%），尽管 1 种期刊的掩蔽成功率明显高于其他期刊（90% 成功掩蔽；95% CI，73%-98%）。众所周知的作者的手稿不太可能被成功掩盖（比值比，0.3；95% CI，0.1-0.8）。当分析仅限于成功屏蔽的手稿时，编辑和作者评估的审稿质量仍然没有差异。结论：像通常的做法一样屏蔽审稿人的作者身份并不能提高审稿质量。由于知名作者的手稿更难屏蔽，而这些手稿可能更有可能从屏蔽中受益，因此无法屏蔽审稿人的知名作者身份可能是导致效果不佳的原因之一。
Context.-All authors may not be equal in the eyes of reviewers. Specifically, well-known authors may receive less objective (poorer quality) reviews. One study at a single journal found a small improvement in review quality when reviewers were masked to author identity.Objectives.-To determine whether masking reviewers to author identity is generally associated with higher quality of review at biomedical journals, and to determine the success of routine masking techniques.Design and Setting.-A randomized controlled trial performed on external reviews of manuscripts submitted to Annals of Emergency Medicine, Annals of Internal Medicine, JAMA, Obstetrics & Gynecology, and Ophthalmology.Interventions.-Two peers reviewed each manuscript. In one study arm, both peer reviewers received the manuscript according to usual masking practice. In the other arm, one reviewer was randomized to receive a manuscript with author identity masked, and the other reviewer received an unmasked manuscript.Main Outcome Measure.-Review quality on a 5-point Likert scale as judged by manuscript author and editor. A difference of 0.5 or greater was considered important.Results.-A total of 118 manuscripts were randomized, 26 to usual practice and 92 to intervention. In the intervention arm, editor quality assessment was complete for 77 (84%) of 92 manuscripts. Author quality assessment was complete on 40 (54%) of 74 manuscripts. Authors and editors perceived no significant difference in quality between masked (mean difference, 0.1; 95% confidence interval [CI], -0.2 to 0.4) and unmasked (mean difference, -0.1;95% CI, -0.5 to 0.4) reviews. We also found no difference in the degree to which the review influenced the editorial decision (mean difference, -0.1;95% CI,-0.3 to 0.3). Masking was often unsuccessful (overall, 68% successfully masked; 95% CI, 58%-77%), although 1 journal had significantly better masking success than others (90% successfully masked; 95% CI, 73%-98%). Manuscripts by generally known authors were less likely to be successfully masked (odds ratio, 0.3; 95% CI, 0.1-0.8). When analysis was restricted to manuscripts that were successfully masked, review quality as assessed by editors and authors still did not differ.Conclusions.-Masking reviewers to author identity as commonly practiced does not improve quality of reviews. Since manuscripts of well-known authors are more difficult to mask, and those manuscripts may be more likely to benefit from masking, the inability to mask reviewers to the identity of well-known authors may have contributed to the lack of effect.