Calibration with Privacy in Peer Review
Calibration with Privacy in Peer Review
复制标题
DOI：
10.1109/isit50566.2022.9834716
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Wenxin Ding;Gautam Kamath;Weina Wang;Nihar B. Shah
中科院分区：
文献类型：
--
作者：
Wenxin Ding;Gautam Kamath;Weina Wang;Nihar B. Shah
这篇论文有资格获得杰克·基尔·沃尔夫学生论文奖。同行评议中的评审者经常被错误地校准：他们可能是严格的、宽松的、极端的、适度的等等。以前已经提出了一些算法来校准评论。然而，这种校准的尝试可能会泄露有关哪个审稿人审阅了哪篇论文的敏感信息。在这篇文章中，我们确定了这个带有隐私的校准问题，并提供了一个基本的构建块来解决它。具体地说，我们在一个简化但具有挑战性的模型下对这个问题进行了理论研究，该模型涉及两名评审员、两篇论文和一个地图计算对手。我们的主要结果建立了隐私(防止对手推断审稿人身份)和效用(接受更好的论文)之间权衡的Pareto前沿，并设计了显式的计算效率高的算法，我们证明了该算法是Pareto最优的。
This paper is eligible for the Jack Keil Wolf ISIT Student Paper Award. Reviewers in peer review are often miscalibrated: they may be strict, lenient, extreme, moderate, etc. A number of algorithms have previously been proposed to calibrate reviews. Such attempts of calibration can however leak sensitive information about which reviewer reviewed which paper. In this paper, we identify this problem of calibration with privacy, and provide a foundational building block to address it. Specifically, we present a theoretical study of this problem under a simplified-yet-challenging model involving two reviewers, two papers, and an MAP-computing adversary. Our main results establish the Pareto frontier of the tradeoff between privacy (preventing the adversary from inferring reviewer identity) and utility (accepting better papers), and design explicit computationally-efficient algorithms that we prove are Pareto optimal.