Learning to Cooperate in Repeated Games

研究重复博弈的一个主要原因是了解自私的玩家如何在没有串通协议的情况下协调他们的行动以取得进步。不幸的是，现有的博弈论模型承认了如此多的结果，以至于无法预测是否会出现协调。分析还假设一个具有无限计算能力和完全预见能力的理性主体。这些假设对均衡模型至关重要，但也相当不现实。这个项目探索了另一种模型，在这种模型中，完全理性的代理被“有限理性”的代理所取代，这些代理只有有限的计算能力，并且不能完全预见其他参与者的策略，他们必须从过去的经验中学习。这种方法被证明可以捕捉学习动态，并允许应用于广泛的重复和动态游戏，这些游戏具有可以影响模型长期结果的“大”参与者。应用范围包括国际债务和具有道德风险的最优增长。更具体地说，该项目研究了两个人重复的游戏，其中每个玩家根据梯度方法学习对手的策略，假设对手根据线性策略进行游戏。此外，每个玩家都人为地添加随机噪音，这些噪音会慢慢消失，以便与对手的策略进行实验。对可行策略没有限制，但每个参与者的预测必须是过去观察的线性函数。选择这类策略的原因是这些策略足够简单，可以很容易地参数化。然后，每个玩家可以使用最小二乘估计来了解对手的策略。代理的偏好也被稍微修改，以便他在选择最佳对策的同时最小化决策过程的复杂性。然后，得到一个递归最小二乘学习模型，其中每个参与者都会随着博弈的进行而更新自己的信念和操作策略。学习动态以概率1收敛，在极限情况下，双方都有相同的估计量。因此，两个参与者的行为是高度相关的，并且结果的极限频率可以通过线性策略中的某种纳什均衡来维持。例如，在囚徒困境博弈中，结果的极限频率必须是合作与背叛的严格凸组合，这意味着参与者必须学会以正概率进行合作。

A primary reason for studying repeated games is to understand how selfish players can coordinate their actions to achieve improvements without a collusive agreement. Unfortunately existing game-theoretic models admit so many outcomes that it is impossible to predict whether coordination will emerge. Also analysis postulates a rational agent who has unbounded computational capability and perfect foresight. These assumptions are critical for equilibrium models but also rather unrealistic. This project explores alternative models in which perfectly rational agents are replaced by `boundedly rational` agents who have only limited computational capabilities, and who cannot perfectly foresee the strategy of other players, which they have to learn from the past experiences. This approach is shown to capture learning dynamics and to permit applications to a wide class of repeated and dynamic games which have a `big` player who can influence the long run outcome of the model. Applications include international debt and optimal growth with moral hazard. More specifically, the project examines two person repeated games where each player learns the opponent's strategy according to the gradient method by assuming that the opponent is playing according to a linear strategy. In addition, each player artificially adds random noise that disappears slowly in order to experiment against the opponent's strategy. No restrictions are imposed on feasible strategies, but the forecast of each player must be a linear function of past observations. The reason for selecting this particular class of strategies is that these strategies are simple enough to be parameterized easily. Then, each player can learn the opponent's strategy using least squares estimation. The agent's preference is also modified slightly so that he is selecting a best response while minimizing the complexity of the decision making process. then, a recursive least squares learning model is obtained, where each player updates his belief as well as his operated game strategy as the game proceeds. The learning dynamics converges with probability 1 and in the limit, both players have an identical estimator. Consequently the behavior of the two players is highly correlated, and the limit frequency of outcomes can be sustained by some Nash equilibrium in linear strategies. In the prisoner's dilemma game, for example, the limit frequency of outcomes must be a strict convex combination of cooperation and defection, which implies that the players must learn to cooperative with positive probability.