On Strategyproof Conference Peer Review
On Strategyproof Conference Peer Review
复制标题
DOI：
10.24963/ijcai.2019/87
复制
发表时间：
2018-06
期刊：
影响因子：
--
通讯作者：
Yichong Xu;H. Zhao;Xiaofei Shi;Nihar B. Shah
中科院分区：
文献类型：
--
作者：
Yichong Xu;H. Zhao;Xiaofei Shi;Nihar B. Shah
我们认为同行评议是在会议环境下进行的，在这种环境下，评议者和提交的文件之间存在冲突。在这种冲突下，审稿人可以通过策略性地操纵自己的评论来影响自己论文的最终排名。目前的同行评审系统并没有设计成防止这种战略行为，除了最低限度的（和不充分的）检查，如不分配论文给冲突的评审员。在这项工作中，我们通过社会选择的透镜来解决这个问题，并提出了一个防策略和有效的同行评议的理论框架。给定的冲突图，满足一个简单的属性，我们首先提出并分析了一个灵活的框架，审查员分配和聚合的评论，保证不仅strategyproofness，但也有一个自然的效率属性（重复性）。我们的框架是基于所谓的分区方法，并可以被视为这种类型的方法，以会议同行评审设置的泛化。然后，我们的经验表明，（作者）冲突图上的必要属性确实满足ICLR-17提交数据，并进一步展示了一个简单的技巧，使分区方法更实际的呼吁下，会议同行评审设置。最后，我们补充我们的积极结果与负面的理论结果，我们证明，在稍强的要求下，它是不可能的任何算法是strategyproof和有效的。
We consider peer review under a conference setting where there are conflicts between the reviewers and the submissions. Under such conflicts, reviewers can manipulate their reviews in a strategic manner to influence the final rankings of their own papers. Present-day peer-review systems are not designed to guard against such strategic behavior, beyond minimal (and insufficient) checks such as not assigning a paper to a conflicted reviewer. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. Given the conflict graph which satisfies a simple property, we first present and analyze a flexible framework for reviewer-assignment and aggregation for the reviews that guarantees not only strategyproofness but also a natural efficiency property (unanimity). Our framework is based on the so-called partitioning method, and can be treated as a generalization of this type of method to conference peer review settings. We then empirically show that the requisite property on the (authorship) conflict graph is indeed satisfied in the ICLR-17 submissions data, and further demonstrate a simple trick to make the partitioning method more practically appealing under conference peer-review settings. Finally, we complement our positive results with negative theoretical results where we prove that under slightly stronger requirements, it is impossible for any algorithm to be both strategyproof and efficient.