AF: Small: Eliciting Accurate and Useful Information from Heterogeneous Agents

这个项目将设计和研究系统，当信息和专业知识分布在许多异构的代理人。 这解决了众包领域的一个核心挑战，这些解决方案可以直接应用于典型的众包问题，包括同行分级，标签垃圾邮件或其他不合规的材料。 此外，这些系统应证明有用的收集和汇总信息的基础科研，声誉系统，和许多决策的情况下，包括采购，产品开发，定价等，因为隐私是一个参与者成本的实例，在这项研究中开发的想法将很容易导入回隐私领域。虽然信息获取领域有许多丰富的应用前景，但必须克服机制设计领域的挑战才能实现其承诺。 现有技术足以在简单的设置中运行机制，但很少努力研究具有不同水平和/或专业领域的异构代理的困难。 特别有问题的是我们称之为分裂问题的问题，其中天真的代理人以系统和非随机的方式犯错误，可以由更专业的代理人预测。 在这样的设置中，专家代理可能会回答错误，因为预计会对幼稚代理进行评分。 这种情况极为成问题，因为它们使上述机制无法向专家征求资料，而这些专家恰恰是最需要联系的人。该项目将开发必要的理论，以推理与异构代理的信息获取机制，解决关键挑战，并设计创新的解决方案来克服这些挑战。 该项目将通过建立新的机制并进行理论分析和实证检验，以表明它们实现了既定目标，从而突破最令人生畏的障碍。 PI将建立在信息获取的丰富和新兴文献的基础上，而无需使用尖端工具和来自各个领域的见解，包括隐私，机制设计，博弈论，适当的评分规则，信息论，概率和学习理论。 如果成功的话，这一建议将通过创建具有可证明的保证与异构代理一起工作的强大机制来改变信息获取领域。

This project will design and study systems for eliciting and aggregating information and expertise when it is distributed across many heterogeneous agents. This addresses a central challenge in the field of crowdsourcing, and these solutions can be directly applied to typical crowdsourcing problems including peer grading, and labeling spam or other non-compliant material. Additionally, these systems should prove useful in gathering and aggregating information in basic scientific research, reputation systems, and many decision-making contexts including purchasing, product development, pricing, etc. Because privacy is an instance of a participant cost, the ideas developed in this research will readily import back to the field of privacy as well. While the field of information elicitation promises many rich applications, challenges in the area of mechanism design must be overcome to fulfill its promise. The current state of the art is sufficient to run mechanisms in simple settings, but very little effort has gone into studying the difficulty of having heterogeneous agents with various levels and/or areas of expertise. Particularly problematic are what we term divisive problems where naïve agents err in a systematic and non-random way that can be predicted by more expert agents. In such a setting, the expert agent may answer incorrectly, in anticipation of being scored against naïve agents. Such occurrences are extremely problematic because they render the above mechanisms incapable of soliciting information from experts, exactly those whom are most important to reach. This project will develop the theory necessary to reason about information elicitation mechanisms with heterogeneous agents, tackle key challenges, and devise innovative solutions to overcome these challenges. This project will break through the most daunting barriers by creating new mechanisms and accompanying theoretical analysis and empirical tests to show that they achieve the stated goals. The PI will build upon the rich and emerging literature in information elicitation without verification using cutting-edge tools and insights from various fields including privacy, mechanism design, game theory, proper scoring rules, information theory, probability, and learning theory. If successful, this proposal will transform the field of information elicitation by creating robust mechanisms having provable guarantees that work with heterogeneous agents.