Is it possible to rank universities using fewer indicators? A study on five international university rankings
Is it possible to rank universities using fewer indicators? A study on five international university rankings
复制标题
DOI：
10.1108/ajim-05-2018-0118
复制
发表时间：
2019-01-21
影响因子：
2.6
通讯作者：
Al, Umut
中科院分区：
文献类型：
--
作者：
Dogan, Guleda;Al, Umut
目的本文的目的是分析多年来研究型国际大学排名（世界大学学术排名（ARWU）、南洋理工大学、学术表现大学排名（URAP）、夸夸雷利·西蒙兹（QS）和圆形大学排名（RUR））中使用的内部指标的相似性，并展示类似指标对2015年总体排名的影响。本研究根据这些目的提出的研究问题如下：如下： 国际大学排名中使用的内部指标相似程度如何？是否可以根据内部指标的相似性对其进行分组？类似的内部指标对整体排名有何影响？设计/方法/途径 所有大学在五个以研究为重点的国际大学排名中所有年份的基于指标的分数，构成本研究第一和第二研究问题的数据集。作者使用多维尺度（MDS）和余弦相似性度量来分析指标的相似性并回答这两个研究问题。以2015年的指标分数和总体排名分数为数据，采用Spearman相关性检验来回答第三个研究问题。结果 分析结果表明，ARWU、NTU 和 URAP 中使用的内部指标高度相似，并且可以根据它们的相似性对它们进行分组。作者还检验了类似指标对这三个排名的2015年总体排名的影响。 NTU和URAP受到省略相似指标的影响最小，这意味着这两个排名有可能使用更少的指标创建与现有总体排名非常相似的总体排名列表。研究限制/影响 CWTS、Mapping Scientific Excellence、Nature Index 和 SCImago 机构排名（截至 2015 年）不包含在本文范围内，因为它们不创建总体排名列表。同样，泰晤士报高等教育、CWUR 和 US 也不包括在内，因为它们没有提供基于指标的分数。 QS无法获取2010年和2011年所需的数据。此外，尽管QS对700多所大学进行了排名，但只能分析2012-2015年排名中的前400所大学。尽管本研究分析了 QS 和 RUR 的数据，但从统计角度无法对这两个排名得出任何结论。实际影响 本研究的结果可能主要由排名机构、政策制定者和决策者考虑。排名机构可以使用结果来审查他们使用的指标，决定在排名中使用哪些指标，并质疑是否有必要继续总体排名。政策制定者和决策者也可能会考虑放弃使用总体排名结果作为其决策和政策的重要输入，从而从这项研究的结果中受益。原创性/价值 本研究首次使用 MDS 和余弦相似性度量来揭示指标的相似性。排名数据存在偏差，需要进行非参数统计分析；因此，使用MDS。该研究涵盖了所有排名年份和排行榜上的所有大学，与文献中类似研究分析较短时间间隔和排行榜上排名靠前大学的数据不同。可以说，本研究在文献综述的基础上，首次对URAP、NTU和RUR的指标内相似性进行分析。
Purpose The purpose of this paper is to analyze the similarity of intra-indicators used in research-focused international university rankings (Academic Ranking of World Universities (ARWU), NTU, University Ranking by Academic Performance (URAP), Quacquarelli Symonds (QS) and Round University Ranking (RUR)) over years, and show the effect of similar indicators on overall rankings for 2015. The research questions addressed in this study in accordance with these purposes are as follows: At what level are the intra-indicators used in international university rankings similar? Is it possible to group intra-indicators according to their similarities? What is the effect of similar intra-indicators on overall rankings? Design/methodology/approach Indicator-based scores of all universities in five research-focused international university rankings for all years they ranked form the data set of this study for the first and second research questions. The authors used a multidimensional scaling (MDS) and cosine similarity measure to analyze similarity of indicators and to answer these two research questions. Indicator-based scores and overall ranking scores for 2015 are used as data and Spearman correlation test is applied to answer the third research question. Findings Results of the analyses show that the intra-indicators used in ARWU, NTU and URAP are highly similar and that they can be grouped according to their similarities. The authors also examined the effect of similar indicators on 2015 overall ranking lists for these three rankings. NTU and URAP are affected least from the omitted similar indicators, which means it is possible for these two rankings to create very similar overall ranking lists to the existing overall ranking using fewer indicators. Research limitations/implications CWTS, Mapping Scientific Excellence, Nature Index, and SCImago Institutions Rankings (until 2015) are not included in the scope of this paper, since they do not create overall ranking lists. Likewise, Times Higher Education, CWUR and US are not included because of not presenting indicator-based scores. Required data were not accessible for QS for 2010 and 2011. Moreover, although QS ranks more than 700 universities, only first 400 universities in 2012-2015 rankings were able to be analyzed. Although QS's and RUR's data were analyzed in this study, it was statistically not possible to reach any conclusion for these two rankings. Practical implications The results of this study may be considered mainly by ranking bodies, policy- and decision-makers. The ranking bodies may use the results to review the indicators they use, to decide on which indicators to use in their rankings, and to question if it is necessary to continue overall rankings. Policy- and decision-makers may also benefit from the results of this study by thinking of giving up using overall ranking results as an important input in their decisions and policies. Originality/value This study is the first to use a MDS and cosine similarity measure for revealing the similarity of indicators. Ranking data is skewed that require conducting nonparametric statistical analysis; therefore, MDS is used. The study covers all ranking years and all universities in the ranking lists, and is different from the similar studies in the literature that analyze data for shorter time intervals and top-ranked universities in the ranking lists. It can be said that the similarity of intra-indicators for URAP, NTU and RUR is analyzed for the first time in this study, based on the literature review.