Designing Optimal Binary Rating Systems
Designing Optimal Binary Rating Systems
复制标题
设计最佳二元评级系统
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Ramesh Johari
中科院分区：
文献类型：
--
作者：
Nikhil Garg;Ramesh Johari
现代在线平台依靠有效的评级系统来了解项目。我们考虑在交易后收集二元反馈的评级系统的优化设计。我们做出三项贡献。首先，我们将评级系统的性能形式化为它恢复项目的真实基础排名（在较大偏差意义上）的速度，同时考虑了项目的基础匹配率和平台的偏好。其次，我们提供了一种有效的算法来计算产生最高性能的二元反馈系统。最后，我们展示了如何利用这一理论视角来凭经验设计一个可实施的、近似最优的评级系统，并使用在 Amazon Mechanical Turk 上收集的真实实验数据来验证我们的方法。
Modern online platforms rely on effective rating systems to learn about items. We consider the optimal design of rating systems that collect binary feedback after transactions. We make three contributions. First, we formalize the performance of a rating system as the speed with which it recovers the true underlying ranking on items (in a large deviations sense), accounting for both items' underlying match rates and the platform's preferences. Second, we provide an efficient algorithm to compute the binary feedback system that yields the highest such performance. Finally, we show how this theoretical perspective can be used to empirically design an implementable, approximately optimal rating system, and validate our approach using real-world experimental data collected on Amazon Mechanical Turk.