AF: Small: Eliciting Accurate and Useful Information from Heterogeneous Agents

该项目将设计和研究当信息和专业知识分布在许多不同的代理中时，用于获取和聚合信息和专业知识的系统。这解决了众包领域的一个核心挑战，这些解决方案可以直接应用于典型的众包问题，包括同行评分、标记垃圾邮件或其他不合规的材料。此外，这些系统应该在基础科学研究、声誉系统以及许多决策环境(包括采购、产品开发、定价等)中收集和汇总信息方面有用。由于隐私是参与者成本的一个实例，因此本研究中提出的想法也将很容易地重新引入隐私领域。虽然信息启发领域有许多丰富的应用前景，但必须克服机制设计领域的挑战才能实现其承诺。目前的技术水平足以在简单的设置中运行机制，但很少有人致力于研究具有各种水平和/或专业领域的异类代理的难度。特别有问题的是我们所说的分裂问题，天真的代理人以系统和非随机的方式出错，这可以被更多的专家代理人预测到。在这种情况下，专家代理可能会回答错误，因为预计会在与天真代理的比赛中得分。这种情况非常有问题，因为它们使上述机制无法从专家那里获取信息，而专家正是最重要的接触对象。该项目将发展必要的理论，以推理不同种类代理人的信息诱导机制，解决关键挑战，并设计出创新的解决方案来克服这些挑战。该项目将突破最令人望而生畏的障碍，创造新的机制，并伴随着理论分析和经验测试，以表明它们实现了所述目标。PI将建立在丰富和新兴的信息启发文献的基础上，而不需要使用尖端工具和来自不同领域的见解，包括隐私、机制设计、博弈论、适当的评分规则、信息论、概率和学习理论。如果成功，这项提议将通过创建具有可证明的保证的可靠机制来改变信息获取领域，这些机制可以与不同的代理一起工作。

This project will design and study systems for eliciting and aggregating information and expertise when it is distributed across many heterogeneous agents. This addresses a central challenge in the field of crowdsourcing, and these solutions can be directly applied to typical crowdsourcing problems including peer grading, and labeling spam or other non-compliant material. Additionally, these systems should prove useful in gathering and aggregating information in basic scientific research, reputation systems, and many decision-making contexts including purchasing, product development, pricing, etc. Because privacy is an instance of a participant cost, the ideas developed in this research will readily import back to the field of privacy as well. While the field of information elicitation promises many rich applications, challenges in the area of mechanism design must be overcome to fulfill its promise. The current state of the art is sufficient to run mechanisms in simple settings, but very little effort has gone into studying the difficulty of having heterogeneous agents with various levels and/or areas of expertise. Particularly problematic are what we term divisive problems where naïve agents err in a systematic and non-random way that can be predicted by more expert agents. In such a setting, the expert agent may answer incorrectly, in anticipation of being scored against naïve agents. Such occurrences are extremely problematic because they render the above mechanisms incapable of soliciting information from experts, exactly those whom are most important to reach. This project will develop the theory necessary to reason about information elicitation mechanisms with heterogeneous agents, tackle key challenges, and devise innovative solutions to overcome these challenges. This project will break through the most daunting barriers by creating new mechanisms and accompanying theoretical analysis and empirical tests to show that they achieve the stated goals. The PI will build upon the rich and emerging literature in information elicitation without verification using cutting-edge tools and insights from various fields including privacy, mechanism design, game theory, proper scoring rules, information theory, probability, and learning theory. If successful, this proposal will transform the field of information elicitation by creating robust mechanisms having provable guarantees that work with heterogeneous agents.