Surrogate Scoring Rules and a Dominant Truth Serum for Information Elicitation

Surrogate Scoring Rules and a Dominant Truth Serum for Information Elicitation
复制标题

用于信息获取的替代评分规则和显性真理血清

DOI：
--
复制
发表时间：
2018
期刊：
arXiv.org
影响因子：
--
通讯作者：
Yiling Chen
Yiling Chen
中科院分区：
--
文献类型：
--
作者：
Yang Liu;Yiling Chen

文献摘要

被引文献

相似文献

严格适当的评分规则（SPSR）被广泛使用时，设计激励机制，以获取私人信息的战略代理人使用实现地面真相信号，他们可以帮助量化的价值。在本文中，我们扩展这样的评分规则设置的机制设计师没有访问地面真相。我们考虑两个这样的设置：（i）当机制设计者可以访问具有已知偏差的地面实况的嘈杂代理版本时的设置;以及（ii）标准对等预测设置，其中代理的报告是机制设计者具有的唯一信息源。 我们引入代理评分规则（SSR）的第一个设置，它使用的噪声地面真相，以评估质量的信息。我们表明，SSR保持严格的适当性SPSR。使用SSR，我们然后开发了一个多任务评分机制-所谓的主导真理血清（ESTA）-以实现严格的适当性时，机制设计者只能访问代理的报告。在比较标准的平衡概念在同行预测，我们表明，在多任务的优势策略，可实现真实性。SSR和SSR的一个显著特征是，尽管缺乏基本事实，但它们量化了信息的质量，就像适当的评分规则对验证设置所做的那样。我们的方法进行了验证，从理论上和经验上收集的数据从真实的人类参与者。
Strictly proper scoring rules (SPSR) are widely used when designing incentive mechanisms to elicit private information from strategic agents using realized ground truth signals, and they can help quantify the value of elicited information. In this paper, we extend such scoring rules to settings where a mechanism designer does not have access to ground truth. We consider two such settings: (i) a setting when the mechanism designer has access to a noisy proxy version of the ground truth, with known biases; and (ii) the standard peer prediction setting where agents' reports are the only source of information that the mechanism designer has. We introduce surrogate scoring rules (SSR) for the first setting, which use the noisy ground truth to evaluate quality of elicited information. We show that SSR preserves the strict properness of SPSR. Using SSR, we then develop a multi-task scoring mechanism -- called dominant truth serum (DTS) -- to achieve strict properness when the mechanism designer only has access to agents' reports. In comparison to standard equilibrium concepts in peer prediction, we show that DTS can achieve truthfulness in a multi-task dominant strategy. A salient feature of SSR and DTS is that they quantify the quality of information despite lack of ground truth, just as proper scoring rules do for the with verification setting. Our method is verified both theoretically and empirically using data collected from real human participants.