Exploring the application of machine learning to expert evaluation of research impact.
Exploring the application of machine learning to expert evaluation of research impact.
复制标题
DOI：
10.1371/journal.pone.0288469
复制
发表时间：
2023
期刊：
影响因子：
3.7
通讯作者：
中科院分区：
文献类型：
--
作者：
本研究的目的是探讨应用机器学习技术对大规模人类专家评价学术研究的影响。使用英国卓越研究框架（2014）中公开的影响案例研究数据，我们训练了五个机器学习模型，这些模型具有一系列定性和定量特征，包括机构、学科、叙事风格（显性和隐性）以及文献计量和政策指标。我们的工作做出了两个重要贡献。基于预测高分和低分影响案例研究的准确性指标，它表明机器学习模型能够处理信息，做出类似于专家评估者的决策。它还提供了对影响案例研究特点的深入了解，如果采用机器学习方法对其进行自动评估，则会受到青睐。实验的结果表明，强大的影响力的制度背景下，选定的指标的叙事风格，以及研究的政策和学术受众的摄取。总体而言，该研究表明，从描述性分析转向预测性分析是有希望的，但建议谨慎使用机器学习评估影响案例研究。
The objective of this study is to investigate the application of machine learning techniques to the large-scale human expert evaluation of the impact of academic research. Using publicly available impact case study data from the UK’s Research Excellence Framework (2014), we trained five machine learning models on a range of qualitative and quantitative features, including institution, discipline, narrative style (explicit and implicit), and bibliometric and policy indicators. Our work makes two key contributions. Based on the accuracy metric in predicting high- and low-scoring impact case studies, it shows that machine learning models are able to process information to make decisions that resemble those of expert evaluators. It also provides insights into the characteristics of impact case studies that would be favoured if a machine learning approach was applied for their automated assessment. The results of the experiments showed strong influence of institutional context, selected metrics of narrative style, as well as the uptake of research by policy and academic audiences. Overall, the study demonstrates promise for a shift from descriptive to predictive analysis, but suggests caution around the use of machine learning for the assessment of impact case studies.
登录
查看更多内容
影响因子：
3.7
作者：
Coutinho E;Schuller B
通讯作者：
Schuller B
影响因子：
9.9
作者：
Flesch, Rudolf
通讯作者：
Flesch, Rudolf
影响因子：
4.2
作者：
Hughes, Tim;Webber, Don;O'Regan, Nicholas
通讯作者：
O'Regan, Nicholas
影响因子：
12.3
作者：
Cui, Zhihua;Xue, Fei;Chen, Jinjun
通讯作者：
Chen, Jinjun
影响因子：
3.7
作者：
Balbuena LD
通讯作者：
Balbuena LD