Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs.
Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs.
复制标题
DOI：
10.1371/journal.pone.0005910
复制
发表时间：
2009-06-18
期刊：
影响因子：
3.7
通讯作者：
Walport M
中科院分区：
文献类型：
--
作者：
Allen L;Jones C;Dolby K;Lynn D;Walport M
将专家评估与文献计量指标作为评估科研论文质量和重要性的工具进行比较。
2005年发表后不久，一批近700篇与维康信托（WT）相关的研究论文的质量和重要性由专家评审员进行了评估；每篇论文都由两名维康信托的专家评审员评审。3年后，我们将这一初始评估与论文影响力的其他衡量标准进行了比较。
发表后不久，687篇研究论文中有62篇（9%）被判定至少描述了“对知识的重大补充”——其中6篇被认为是“里程碑式”的论文。总体而言，3年后，专家评估与以引用次数和F1000评级衡量的影响力之间存在很强的正相关。然而，也有一些重要的例外情况，这表明文献计量指标单独作为研究质量和重要性的衡量标准可能是不够的，尤其是在评估单篇论文或少量研究出版物时。
当试图评估研究论文的质量和重要性时，我们发现仅仅依赖文献计量指标会使我们错过经专家评审判定包含重要结果的论文。特别是，一些被专家高度评价的论文在发表后的前三年引用次数并不多。将研究论文质量和重要性的专家同行评审与更定量的指标（如引用分析）联系起来的工具，将是研究评估领域有价值的补充。
To compare expert assessment with bibliometric indicators as tools to assess the quality and importance of scientific research papers. Shortly after their publication in 2005, the quality and importance of a cohort of nearly 700 Wellcome Trust (WT) associated research papers were assessed by expert reviewers; each paper was reviewed by two WT expert reviewers. After 3 years, we compared this initial assessment with other measures of paper impact. Shortly after publication, 62 (9%) of the 687 research papers were determined to describe at least a ‘major addition to knowledge’ –6 were thought to be ‘landmark’ papers. At an aggregate level, after 3 years, there was a strong positive association between expert assessment and impact as measured by number of citations and F1000 rating. However, there were some important exceptions indicating that bibliometric measures may not be sufficient in isolation as measures of research quality and importance, and especially not for assessing single papers or small groups of research publications. When attempting to assess the quality and importance of research papers, we found that sole reliance on bibliometric indicators would have led us to miss papers containing important results as judged by expert review. In particular, some papers that were highly rated by experts were not highly cited during the first three years after publication. Tools that link expert peer reviews of research paper quality and importance to more quantitative indicators, such as citation analysis would be valuable additions to the field of research assessment and evaluation.
登录
查看更多内容
影响因子：
3.9
作者：
Adams, Jonathan;Gurney, Karen;Marshall, Stuart
通讯作者：
Marshall, Stuart
影响因子：
50.5
作者：
Berghmans, T;Meert, AP;Sculier, JP
通讯作者：
Sculier, JP
DOI：
10.1073/pnas.0407743101
发表时间：
2004-12-21
影响因子：
11.1
作者：
Giles, CL;Councill, IG
通讯作者：
Councill, IG
影响因子：
56.9
作者：
Wuchty, Stefan;Jones, Benjamin F.;Uzzi, Brian
通讯作者：
Uzzi, Brian
影响因子：
7.2
作者：
Papatheodorou, Stefania I.;Trikalinos, Thomas A.;Ioannidis, John P. A.
通讯作者：
Ioannidis, John P. A.