An Experimental Study of Learning in Repeated Games Using Real Beliefs

近年来，博弈论学家和实验经济学家将大量注意力集中在人们在反复玩简单矩阵游戏时是如何学习的问题上。一些研究者专注于强化学习，即人们通过回顾自己的经验和过去的成功经验来学习，而另一些研究者则专注于信念学习，并通过回顾过去来更新对对手未来行为的信念。还有一些人，选择这两个模型的最佳特性（在其他事物中），以一种已被证明非常成功的方法。然而，在所有这些研究中，有一个假设是，虽然过去的行为和回报是可观察的，但信念是不可观察的，因此必须通过代理和推断来表示。这里提出的研究试图利用实验室技术，通过使用“适当的评分规则”直接引出实验对象的信念，该规则为实验对象提供了如实报告其信念的激励。因此，本研究提出了一种信念学习的研究，其中所有相关变量都是可观察的；也就是说，它用真实的信念来研究信念学习。在一篇刚刚完成的关于具有真实信念的2乘2常数和博弈的论文中，主要研究人员得出了两个广泛的结论。首先，在人们对某些信念做出最佳反应的假设下，他们发现陈述的信念比研究的其他信念更符合数据（古诺，虚构的游戏，更普遍的是，伽马加权经验分布）。其次，在假设人们通过一些信念产生的逻辑信念学习规则来选择策略的情况下，我们再次发现陈述的信念最适合数据。这里提出的研究在四个不同的方向上扩展了我们以前的工作。首先，我们提出调查是否我们的数据可以更好地解释强化学习模型。这种比较很有趣，因为我们打算将“最佳”强化学习模型的性能与我们陈述的信念学习模型进行比较，该模型的性能优于我们所比较的所有其他基于历史的信念学习模型。换句话说，我们打算比较最佳强化学习模型和最佳信念学习模型。其次，利用我们的引出信念，我们打算看看，在重复的矩阵博弈中，受试者是否收敛于“信念均衡”，在这种均衡中，每个受试者都相信他们的对手的行为是由理论均衡所决定的。这将允许对博弈论均衡概念的真实性进行额外的测试，因为仅依靠行动来支持理论可能会产生误导。简而言之，在行为方面看似不平衡的东西，在信念方面可能仍然是平衡的。第三，我们构建了新的信念形成理论模型，并使用我们之前和提出的实验产生的数据对它们进行了测试。最后，我们将扩展我们研究的游戏集，超越两人常和游戏，以确定当我们改变游戏主体的正常形式时，信念形成过程是如何变化的。

In recent years game theorists and experimental economists have focused a great deal of attention on the question of how people learn when repeatedly playing a simple matrix game. While some, investigators focus on reinforcement learning in which people learn by looking back at their experience and seeing what has been successful in the past, others, focus on belief learning and look to the past to update beliefs about their opponent's future action. Still others, select the best features of both of these models (among other things) in an approach that has proven to be remarkable successful. In all of this research, however, there is an assumption that while past actions and payoffs are observable, beliefs are unobservable and therefore must be represented by proxies and inferred. The research proposed here attempts to take advantage of laboratory techniques by directly eliciting the beliefs of subjects in our experiments using a "proper scoring rule" which provides subjects with an incentive to report their beliefs truthfully. As a result, this proposed research presents, an investigation of belief learning in which all relevant variables are observable; i.e., it studies belief learning with real beliefs., In a just completed paper on two-by-two constant-sum games with real beliefs, the principal investigators, come to two broad conclusions. First, under the assumption that people best-respond to some beliefs, they find that stated beliefs fit the data better than the other beliefs studied (Cournot, fictitious play and more generally, the class of gamma-weighted empirical distributions). Second, under the assumption that people choose strategies via a logistic belief learning rule generated by some beliefs, we again find that stated beliefs fit the data the best. The research proposed here extends our previous work in four different directions. First we propose to investigate whether our data can better be explained by a reinforcement learning model. This comparison is interesting since we intend to compare the performance of "the best" reinforcement learning model to our stated belief learning model, a model which has out-performed all of the other history based belief learning models we have compared it to. In other words, we intend to compare the best reinforcement learning model to the best belief learning model. Second, using our elicited beliefs, we intend to see if, in the repeated play of matrix games, subjects converge to an "equilibrium in beliefs" in which each subject believes their opponent to behave as dictated by the equilibrium of the theory. This will allow an additional test of the veracity of game theoretical equilibrium notions since relying on actions alone to support the theory can be misleading. In short, what may appear to be disequilibrium with respect to actions may still be an equilibrium with respect to beliefs. Third, we construct new theoretical models of belief formation and test them using the data generated by our previous and proposed experiments. Finally, we will expand the set of games we investigate beyond the two-person constant sum games to determine how the belief formation process changes as we change the normal form of the game subjects play.