Learning to Cooperate in Repeated Games

研究重复博弈的一个主要原因是了解自私的参与者如何在没有共谋协议的情况下协调他们的行动以实现改进。 不幸的是，现有的博弈论模型承认如此多的结果，以至于不可能预测是否会出现协调。 分析还假设一个理性的代理人谁拥有无限的计算能力和完美的远见。 这些假设对均衡模型来说是至关重要的，但也相当不现实。 该项目探讨了替代模型，其中完全理性的代理人被“有限理性”代理人所取代，这些代理人只有有限的计算能力，并且无法完全预见其他玩家的策略，他们必须从过去的经验中学习。 这种方法被证明是捕捉学习动态，并允许应用程序广泛的重复和动态的游戏，有一个“大”的球员谁可以影响长期运行的结果模型。 申请包括国际债务 和道德风险下的最优增长 更具体地说，该项目研究了两个人的重复游戏，其中每个玩家根据梯度法学习对手的策略，假设对手是根据线性策略进行游戏。 此外，每个玩家都人为地添加了随机噪音，这些噪音会慢慢消失，以便对对手的策略进行实验。 对可行策略没有限制，但每个参与者的预测必须是过去观察的线性函数。 选择这类特殊策略的原因是这些策略足够简单，可以很容易地参数化。 然后，每个玩家可以使用最小二乘估计来学习对手的策略。 代理人的偏好也略有修改，使他选择最佳对策，同时最大限度地减少决策过程的复杂性。 然后，得到递归最小二乘学习模型，其中每个参与者随着游戏的进行更新他的信念以及他的操作游戏策略。 学习动态以概率1收敛，并且在极限下，两个参与者具有相同的估计量。 因此，两个参与者的行为是高度相关的，并且结果的极限频率可以通过线性策略中的某些纳什均衡来维持。 例如，在囚徒困境博弈中，结果的极限频率必须是合作和背叛的严格凸组合，这意味着参与者必须学会以正概率合作。

A primary reason for studying repeated games is to understand how selfish players can coordinate their actions to achieve improvements without a collusive agreement. Unfortunately existing game-theoretic models admit so many outcomes that it is impossible to predict whether coordination will emerge. Also analysis postulates a rational agent who has unbounded computational capability and perfect foresight. These assumptions are critical for equilibrium models but also rather unrealistic. This project explores alternative models in which perfectly rational agents are replaced by `boundedly rational` agents who have only limited computational capabilities, and who cannot perfectly foresee the strategy of other players, which they have to learn from the past experiences. This approach is shown to capture learning dynamics and to permit applications to a wide class of repeated and dynamic games which have a `big` player who can influence the long run outcome of the model. Applications include international debt and optimal growth with moral hazard. More specifically, the project examines two person repeated games where each player learns the opponent's strategy according to the gradient method by assuming that the opponent is playing according to a linear strategy. In addition, each player artificially adds random noise that disappears slowly in order to experiment against the opponent's strategy. No restrictions are imposed on feasible strategies, but the forecast of each player must be a linear function of past observations. The reason for selecting this particular class of strategies is that these strategies are simple enough to be parameterized easily. Then, each player can learn the opponent's strategy using least squares estimation. The agent's preference is also modified slightly so that he is selecting a best response while minimizing the complexity of the decision making process. then, a recursive least squares learning model is obtained, where each player updates his belief as well as his operated game strategy as the game proceeds. The learning dynamics converges with probability 1 and in the limit, both players have an identical estimator. Consequently the behavior of the two players is highly correlated, and the limit frequency of outcomes can be sustained by some Nash equilibrium in linear strategies. In the prisoner's dilemma game, for example, the limit frequency of outcomes must be a strict convex combination of cooperation and defection, which implies that the players must learn to cooperative with positive probability.