Comparison of bibliographic data sources: Implications for the robustness of university rankings
Comparison of bibliographic data sources: Implications for the robustness of university rankings
复制标题
DOI：
10.1162/qss_a_00031
复制
发表时间：
2020-03-01
影响因子：
6.4
通讯作者：
Ozaygen, Alkim
中科院分区：
文献类型：
--
作者：
(karl) Huang, Chun-Kai;Neylon, Cameron;Ozaygen, Alkim
对大学的评价越来越多地以其产出为基础。这些排名往往被转换成简单的、有争议的排名，对招聘、收入和声望有重大影响。此类评估通常依赖于单个数据源来定义大学的输出集。然而，很少有研究探讨了不同的数据来源及其对机构规模的指标和排名的影响。我们通过在机构层面上对Web of Science（WoS）、Scopus和Microsoft Academic（MSA）进行详细的书目比较来解决这一差距，并对15所大学进行手动分析。我们进一步构建了两个简单的排名的基础上引用计数和开放获取状态。我们的研究结果表明，不同数据库之间存在显着差异。这些差异导致了大学排名的急剧变化，这在非英语大学和国际大学排名中排名靠前的大学中最为普遍。总体而言，MSA比Scopus和WoS有更大的覆盖范围，但不太完整的从属关系元数据。我们建议，强大的评估措施需要考虑的影响，选择数据源，并建议一种方法，从多个来源的数据集成，以提供一个更强大的数据集。
Universities are increasingly evaluated on the basis of their outputs. These are often converted to simple and contested rankings with substantial implications for recruitment, income, and perceived prestige. Such evaluation usually relies on a single data source to define the set of outputs for a university. However, few studies have explored differences across data sources and their implications for metrics and rankings at the institutional scale. We address this gap by performing detailed bibliographic comparisons between Web of Science (WoS), Scopus, and Microsoft Academic (MSA) at the institutional level and supplement this with a manual analysis of 15 universities. We further construct two simple rankings based on citation count and open access status. Our results show that there are significant differences across databases. These differences contribute to drastic changes in rank positions of universities, which are most prevalent for non-English-speaking universities and those outside the top positions in international university rankings. Overall, MSA has greater coverage than Scopus and WoS, but with less complete affiliation metadata. We suggest that robust evaluation measures need to consider the effect of choice of data sources and recommend an approach where data from multiple sources is integrated to provide a more robust data set.