The Quality Assist: A Technology-Assisted Peer Review Based on Citation Functions to Predict the Paper Quality
The Quality Assist: A Technology-Assisted Peer Review Based on Citation Functions to Predict the Paper Quality
复制标题
DOI：
10.1109/access.2022.3225871
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
Setio Basuki;Masatoshi Tsuchiya
中科院分区：
文献类型：
--
作者：
Setio Basuki;Masatoshi Tsuchiya
本研究旨在开发论文质量评估的预测模型，以支持技术辅助的同行评审。预测技术旨在减轻评审负担，这正在成为当今论文提交过程中的一个关键问题。然而，大多数现有的关于该主题的作品都是通过引用审稿人的意见来构建的，这被认为是不公平的，并且不适用于减轻审稿负担。因此，我们的预测方法仅依赖于从论文中提取的特征来解决这个问题。该方法涵盖以下三个任务：两个是分类任务，一个是回归任务。分类任务预测最终的审稿决定（接受-拒绝）并估计论文质量（好-差），而回归任务则预测审稿分数。此外，分类和回归任务是使用三个主要特征来实现的，即基于引文函数标签方案开发的引文句子特征、通过将引文函数标签应用于非引文文本创建的常规句子特征以及通过识别引文来源构建的基于参考的特征。此外，对 2017-2020 年国际学习表征会议获得的数据集进行的分类实验表明，我们的方法在好-差任务中比接受-拒绝任务更有效，最佳准确率分别为 0.75 和 0.73。此外，仅使用引用句子特征，我们还达到了令人满意的 0.99 召回率，以便在好-差任务中获得尽可能多的好论文。我们的回归实验表明，预测平均评论评分的最佳结果高于个人评论评分，均方根误差 (RMSE) 分别为 1.34 和 1.71。
This study aims to develop a prediction model for paper quality assessment to support technology-assisted peer review. The prediction technique is intended to reduce the review burden, which is becoming a critical issue in today’s paper submission process. However, most existing works on this topic were built by involving the reviewers’ comments, which is considered unfair and inapplicable for reducing the review burden. Therefore, our prediction method relies only on features extracted from the paper to address this issue. The method covers three tasks as follows: two are classification tasks and one is a regression task. The classification tasks predict the final review decision (accepted-rejected) and estimate the paper quality (good-poor), while a regression task predicts the review scores. Additionally, the classification and regression tasks are implemented using three main features i.e., citing sentence features developed based on the labeling scheme of citation functions, regular sentence features created by applying the label of citation functions to non-citation text, and reference-based features constructed by identifying the source of citations. Furthermore, the classification experiments on the dataset obtained from the International Conference on Learning Representations 2017–2020 showed that our methods are more effective in the good-poor task than the accepted-rejected task by demonstrating the best accuracy of 0.75 and 0.73, respectively. Moreover, we also reached a satisfactory recall of 0.99 using only the citing sentence features to obtain as many good papers as possible in the good-poor task. Our regression experiments indicate that the best result in predicting the average review score is higher than the individual review score by showing Root Mean Square Error (RMSE) of 1.34 and 1.71, respectively.