Mitigating Skewed Bidding for Conference Paper Assignment
Mitigating Skewed Bidding for Conference Paper Assignment
复制标题
DOI：
10.48550/arxiv.2303.00435
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Inbal Rozencweig;R. Meir;Nick Mattei;Ofra Amir
中科院分区：
文献类型：
--
作者：
Inbal Rozencweig;R. Meir;Nick Mattei;Ofra Amir
人工智能及相关领域会议论文投稿数量的爆炸式增长，凸显了改进同行评审流程诸多方面的必要性，尤其是论文与评审人员的匹配。近期的研究认为，改进这种匹配的关键在于修改“投标阶段”本身的一些方面，以确保对论文的投标集合是平衡的，尤其要避免“孤儿论文”，即那些没有收到任何投标的论文。为了理解并缓解这一问题，我们开发了一个灵活的投标平台来测试对投标流程的调整。利用这个平台，我们在一个中等规模的国际研讨会的投标阶段进行了一项现场实验，比较了两种投标方法。我们还通过在亚马逊 Mechanical Turk 上进行的对照实验，进一步研究了影响投标的各种因素，特别是论文呈现的顺序\cite{cabanac2013capitalizing,fiez2020super}；以及关于论文需求的信息\cite{meir2021market}。我们的研究结果表明，一些可以添加到任何现有平台的简单调整，可能会显著减少投标的偏差，从而改善评审人员和会议组织者的分配情况。
The explosion of conference paper submissions in AI and related fields, has underscored the need to improve many aspects of the peer review process, especially the matching of papers and reviewers. Recent work argues that the key to improve this matching is to modify aspects of the \emph{bidding phase} itself, to ensure that the set of bids over papers is balanced, and in particular to avoid \emph{orphan papers}, i.e., those papers that receive no bids. In an attempt to understand and mitigate this problem, we have developed a flexible bidding platform to test adaptations to the bidding process. Using this platform, we performed a field experiment during the bidding phase of a medium-size international workshop that compared two bidding methods. We further examined via controlled experiments on Amazon Mechanical Turk various factors that affect bidding, in particular the order in which papers are presented \cite{cabanac2013capitalizing,fiez2020super}; and information on paper demand \cite{meir2021market}. Our results suggest that several simple adaptations, that can be added to any existing platform, may significantly reduce the skew in bids, thereby improving the allocation for both reviewers and conference organizers.