HCC: Small: Incentive-Compatible Machine Learning

计算机制设计在为具有多个自利主体的系统中的协调决策提供了一个理论和实践框架方面取得了很大的成功。机制设计理论中引入的一个关键性质是“激励相容”，即即使在战略上相互依存的环境中，真实地披露关于不同结果的偏好的私人信息也是代理人的最佳利益。然而，对于使用激励机制来协调计算过程的关注相对较少，在这种情况下，计算的输入是跨代理分布的。这个项目的重点是激励相容学习的问题，其中代理的私人信息代表“训练数据”，设计目标是允许系统从这些信息所代表的分布式经验中集体学习。研究寻找一种机制，促进在自利代理下的学习与在合作代理下一样有效，并在其他情况下理解这是不可能的。它将在机械设计理论和计算机科学之间架起一座新的桥梁。该研究围绕三个主题展开：(1)激励相容强化学习，其设计目标是当每个智能体具有关于状态空间某一子集的奖励的私有信息并且可能错误报告它们时，快速学习整个状态空间的最优策略；(2)激励相容监督学习，其中每个智能体具有一组标记的训练实例的私有知识，其设计目标是学习一个假设，该假设最小化全局误差，尽管智能体能够错误报告训练数据；(3)激励相容的信息聚合，其中每个智能体对某些不确定事件发生的概率有一个主观信念，并参与了一种利用其信息的机制，设计目标是实现在线信息聚合，而不考虑智能体的内在自利。该项目有可能为社会、工程和商业环境带来广泛的溢出效益，在这些环境中，学习是与自私的代理人一起进行的。

Computational mechanism design has been highly successful in providing a theoretical and practical framework for coordinated decision making in systems with multiple, self-interested agents. A key property introduced within mechanism design theory is that of "incentive-compatibility," namely that it is an agent's best interest to truthfully reveal private information about its preferences for different outcomes even in settings of strategic interdependence. However, there has been relatively little attention given to the use of incentive mechanisms for the purpose of coordinating computational processes, where the inputs to the computation are distributed across agents. The focus of this project is on the problem of incentive-compatible learning, where the private information of agents represents "training data" and the design goal is to allow the system to collectively learn from the distributed experience that this information represents.The research seeks mechanisms that promote learning with self-interested agents that is just as effective as it would be with cooperative agents, and to otherwise understand when this is not possible. It will provide a new bridge between mechanism design theory and computer science. The research is centered around three themes: (1) incentive-compatible reinforcement learning, where the design goal is to quickly learn an optimal policy for the entire state space when each agent has private information about the rewards for some subset of the state space and may misreport them; (2) incentive-compatible supervised learning, where each agent has private knowledge of a set of labeled training examples and the design goal is to learn a hypothesis that minimizes global error despite agents' ability to misreport training data; (3) incentive-compatible information aggregation, where each agent has a subjective belief about the probability of some uncertain events and participates in a mechanism to capitalize on its information, and the design goal is to achieve online aggregation of information despite the intrinsic self-interest of agents. This project has the potential for broad spillover benefits to societal, engineering, and business settings where learning is performed with self-interested agents.