AF: Small: Eliciting Accurate and Useful Information from Heterogeneous Agents

这个项目将设计和研究系统，当信息和专业知识分布在许多不同的代理中时，它将被激发和聚集。这解决了众包领域的一个核心挑战，这些解决方案可以直接应用于典型的众包问题，包括同行评分，标记垃圾邮件或其他不合规的材料。此外，这些系统应该在基础科学研究、声誉系统和许多决策环境（包括采购、产品开发、定价等）中收集和汇总信息方面被证明是有用的。由于隐私是参与者成本的一个实例，本研究中发展的思想也很容易引入到隐私领域。虽然信息获取领域有许多丰富的应用前景，但要实现其前景，必须克服机制设计方面的挑战。目前的技术水平足以在简单的设置中运行机制，但是很少有人努力研究具有不同级别和/或专业领域的异构代理的困难。特别有问题的是我们所说的分裂问题，naïve代理以系统和非随机的方式出错，可以被更专业的代理预测。在这种情况下，专家代理可能会错误地回答，因为预期会被naïve代理得分。这种情况极为成问题，因为它们使上述机制无法向专家征求信息，而这些专家恰恰是最重要的接触对象。该项目将发展必要的理论来推理异构代理的信息激发机制，解决关键挑战，并设计创新的解决方案来克服这些挑战。该项目将通过建立新机制和伴随的理论分析和实证测试来突破最艰巨的障碍，以表明它们实现了既定目标。PI将以丰富的新兴文献为基础，利用来自隐私、机制设计、博弈论、适当评分规则、信息论、概率论和学习理论等各个领域的前沿工具和见解，进行无需验证的信息获取。如果成功，该建议将通过创建具有可证明的保证的健壮机制来改变信息获取领域，这些机制可以与异构代理一起工作。

This project will design and study systems for eliciting and aggregating information and expertise when it is distributed across many heterogeneous agents. This addresses a central challenge in the field of crowdsourcing, and these solutions can be directly applied to typical crowdsourcing problems including peer grading, and labeling spam or other non-compliant material. Additionally, these systems should prove useful in gathering and aggregating information in basic scientific research, reputation systems, and many decision-making contexts including purchasing, product development, pricing, etc. Because privacy is an instance of a participant cost, the ideas developed in this research will readily import back to the field of privacy as well. While the field of information elicitation promises many rich applications, challenges in the area of mechanism design must be overcome to fulfill its promise. The current state of the art is sufficient to run mechanisms in simple settings, but very little effort has gone into studying the difficulty of having heterogeneous agents with various levels and/or areas of expertise. Particularly problematic are what we term divisive problems where naïve agents err in a systematic and non-random way that can be predicted by more expert agents. In such a setting, the expert agent may answer incorrectly, in anticipation of being scored against naïve agents. Such occurrences are extremely problematic because they render the above mechanisms incapable of soliciting information from experts, exactly those whom are most important to reach. This project will develop the theory necessary to reason about information elicitation mechanisms with heterogeneous agents, tackle key challenges, and devise innovative solutions to overcome these challenges. This project will break through the most daunting barriers by creating new mechanisms and accompanying theoretical analysis and empirical tests to show that they achieve the stated goals. The PI will build upon the rich and emerging literature in information elicitation without verification using cutting-edge tools and insights from various fields including privacy, mechanism design, game theory, proper scoring rules, information theory, probability, and learning theory. If successful, this proposal will transform the field of information elicitation by creating robust mechanisms having provable guarantees that work with heterogeneous agents.