Context-Aware Reviewer Assignment for Trust Enhanced Peer Review.
Context-Aware Reviewer Assignment for Trust Enhanced Peer Review.
复制标题
增强信任同行评审的情境感知评审员分配。
DOI：
10.1371/journal.pone.0130493
复制
发表时间：
2015
期刊：
影响因子：
3.7
通讯作者：
Wu X
中科院分区：
文献类型：
--
作者：
Li L;Wang Y;Liu G;Wang M;Wu X
评审员分配是同行评审系统的关键，例如同行评审的研究会议或同行评审的资金申请，其有效性是所有学者都非常关注的问题。然而，现有的同行评议制度在评审员指派过程中存在一些问题。例如，有些评审员比其他人严格得多，导致了不公平的最终决定，即一些质量较好的投稿(即论文或申请)被拒绝。在本文中，我们提出了一种上下文感知的审阅者分配方法，用于信任增强型同行评审。更具体地说，在我们的方法中，我们首先考虑审稿人在研究领域的特定专长，以及审稿人和作者之间的机构相关性和合作关系，以便具有适当专长的审稿人被分配到相应的投稿中，而不会出现潜在的利益冲突。此外，我们提出了一种新的交叉分配范式，并且交叉分配评审者，以避免将一组严格的评审者或一组宽松的评审者分配给同一投稿。更重要的是，我们提出了一种学术背景感知的专家相关性导向的评审者交叉分配方法(Constant)，该方法旨在基于所有评审者的评分来有效地估计提交的“真实”评分，即使不存在关于严格评审者和宽松评审者的分布的先验知识。实验表明，与现有方法相比，无论严格评审者和宽松评审者的分布是什么，我们提出的一致性方法都不太可能同时为一篇投稿分配多个严格评审者或宽松评审者，并显著降低了严格评审者和宽松评审者对评分的影响，从而导致信任增强的同行评审和选择。
Reviewer assignment is critical to peer review systems, such as peer-reviewed research conferences or peer-reviewed funding applications, and its effectiveness is a deep concern of all academics. However, there are some problems in existing peer review systems during reviewer assignment. For example, some of the reviewers are much more stringent than others, leading to an unfair final decision, i.e., some submissions (i.e., papers or applications) with better quality are rejected. In this paper, we propose a context-aware reviewer assignment for trust enhanced peer review. More specifically, in our approach, we first consider the research area specific expertise of reviewers, and the institution relevance and co-authorship between reviewers and authors, so that reviewers with the right expertise are assigned to the corresponding submissions without potential conflict of interest. In addition, we propose a novel cross-assignment paradigm, and reviewers are cross-assigned in order to avoid assigning a group of stringent reviewers or a group of lenient reviewers to the same submission. More importantly, on top of them, we propose an academic CONtext-aware expertise relevanCe oriEnted Reviewer cross-assignmenT approach (CONCERT), which aims to effectively estimate the “true” ratings of submissions based on the ratings from all reviewers, even though no prior knowledge exists about the distribution of stringent reviewers and lenient reviewers. The experiments illustrate that compared with existing approaches, our proposed CONCERT approach can less likely assign more than one stringent reviewers or lenient reviewers to a submission simultaneously and significantly reduce the influence of ratings from stringent reviewers and lenient reviewers, leading to trust enhanced peer review and selection, no matter what kind of distributions of stringent reviewers and lenient reviewers are.
登录
查看更多内容
DOI：
10.1002/asi.20677
发表时间：
2007-11-01
影响因子：
--
作者：
Meho, Lokman I.;Yang, Kiduk
通讯作者：
Yang, Kiduk
影响因子：
3.9
作者：
Hirsch, J. E.
通讯作者：
Hirsch, J. E.
影响因子：
2.4
作者：
Wood, M;Roberts, M;Howell, B
通讯作者：
Howell, B
影响因子：
3.9
作者：
Bartneck C;Kokkelmans S
通讯作者：
Kokkelmans S
影响因子：
3.7
作者：
Bornmann L;Mutz R;Daniel HD
通讯作者：
Daniel HD