The Toronto Paper Matching System: An automated paper-reviewer assignment system
The Toronto Paper Matching System: An automated paper-reviewer assignment system
复制标题
DOI：
--
复制
发表时间：
2013-05
期刊：
影响因子：
--
通讯作者：
Laurent Charlin;R. Zemel
中科院分区：
文献类型：
--
作者：
Laurent Charlin;R. Zemel
会议组织者最重要的任务之一是将论文分配给审稿人。审稿人对论文的评估是确定会议议程的关键一步，从某种意义上来说也决定了一个领域的方向。然而，这不是一项简单的任务：大型会议通常需要将数百篇论文分配给数百名审稿人，而时间限制使得这项任务不可能由一个人完成。此外，还必须考虑其他约束，例如审阅者负载，从而防止流程完全分散。我们构建了系统的第一个版本，为 NIPS 2010 会议建议审稿人分配，随后在 2012 年发布了一个版本，将我们的系统与 Microsoft 流行的会议管理工具包 (CMT) 更好地集成。从那时起，我们的系统已被机器学习和计算机视觉社区的领先会议广泛采用。本文提供了该系统的概述，总结了我们一直在使用的学习模型和评估方法，以及一些最近的进展和未解决的问题。
One of the most important tasks of conference organizers is the assignment of papers to reviewers. Reviewers’ assessments of papers is a crucial step in determining the conference program, and in a certain sense to shape the direction of a field. However this is not a simple task: large conferences typically have to assign hundreds of papers to hundreds of reviewers, and time constraints make the task impossible for one person to accomplish. Furthermore other constraints, such as reviewer load have to be taken into account, preventing the process from being completely distributed. We built the first version of a system to suggest reviewer assignments for the NIPS 2010 conference, followed, in 2012, by a release that better integrated our system with Microsoft’s popular Conference Management Toolkit (CMT). Since then our system has been widely adopted by the leading conferences in both the machine learning and computer vision communities. This paper provides an overview of the system, a summary of learning models and methods of evaluation that we have been using, as well as some of the recent progress and open issues.