Collaborative Research: Disentangling Exploration from Exploitation

这组合作奖资助了一个经济理论项目。该项目将开发一种新的方法来分析人们如何在尝试新体验或继续长期选择之间做出决定。新方法适用于许多现有模型不能很好地解释的现实世界情况。例如，投资者可能投资于一只股票，同时收集有关另一种投资回报的信息。政策制定者可能会在实施现有政策的同时探索新的替代方案。员工可能会在继续当前工作的同时寻找一份新工作。该模型适用于这样的情况，即个人在多个奖励未知的项目中进行选择，并可以投资于一个项目以获得奖励，同时还可以观察不同项目的奖励。这个新模型是经济学和统计学中一大类模型的替代方案，这些模型研究的是一种不同的实验问题，通常被称为“强盗问题”。在土匪问题中，决策者只能通过投资来学习。她只获得关于她选择的项目的信息：她学到的与她赚到的有关。结果是众所周知的探索和开发之间的权衡：决策者可能希望在低回报的项目中投资，因为它提供了有价值的学习。现有的土匪组模型得到了广泛的应用。由这笔拨款资助的新研究可能会导致对类似广泛应用的更深入理解，包括可以增强国家安全和加强美国经济的应用。研究小组将考虑三种与经典实验框架形成鲜明对比的环境。首先，当个体进行实验时，最终总会发现最优方案。这与经典贴现实验中不完全学习的经典结果形成了对比。然而，由于不需要存在简单索引（例如在经典框架中表征解的Gittins索引），因此行为比在标准实验环境中更为复杂。第二，由相似成员组成的团队不再受制于搭便车的问题：当观察和开发分离时，团队实现了有效的实验。该团队还将分析在标准设置中无法解决的问题。具体来说，该项目探索了一个委托环境，其中有两个代理：执行者（执行者）选择要实施的项目，而情报机构（观察者）选择要研究的项目。由于这两个主体的目标可能不一致，委托选择可能产生最小或最大的实验，这取决于行动者和观察者之间接触的性质。最终，该项目有可能发现实验的新特性，这些特性可以应用于各种应用程序并针对数据进行测试。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This group of collaborative awards funds a project in economic theory. The project will develop a new approach to analyze how people make decisions between trying new experiences or continuing with long standing choices. The new approach applies to many real world situations that are not well explained with existing models. For instance, an investor might invest in one stock while gathering information on the returns to a different investment. A policymaker might explore new alternatives while implementing existing policy. An employee might search for a new job while continuing her current employment. The model applies to these situations where an individual chooses between multiple projects with unknown rewards and can invest in one to earn a reward while also observing the rewards of different projects. The new model is an alternative to a large class of models from economics and statistics that has studied a different kind of experimentation problem, often referred to as the bandit problem. In a bandit problem, the decision maker can only learn through investing. She gains information only about the project that she chooses: what she learns is linked to what she earns. The result is a well-known trade-off between exploration and exploitation: the decision - maker may desire investment in a low reward project because it offers valuable learning. The existing group of bandit models have been widely applied. The new research funded by this grant could result in deeper understanding of a similarly wide range of applications, including applications that could enhance national security and strengthen the US economy. The research team will consider three environments that offer a particularly stark contrast with the classical experimentation framework. First, when experimentation is conducted by individuals, the optimal alternative is always eventually discovered. This contrasts with the classical result of incomplete learning in classical experimentation with discounting. However, behavior is more complex than in standard experimentation environments since a simple index (such as the Gittins index,which characterizes solutions in the classical framework) need not exist. Second, teams composed of similar members are no longer subject to free-rider problems: when observation and exploitation are separated, teams achieve efficient experimentation. The team will also analyze questions that cannot be addressed in the standard setting. Specifically, the project explores a delegation environment in which there are two agents: the executive (Doer), who chooses the project to be implemented, and the intelligence agency (Observer), who chooses what project to look at. Because the objectives of these two agents may be misaligned, delegated choices may yield either minimal or maximal experimentation depending on the nature of contact between the doer and the observer. Ultimately, the project has the potential to uncover novel features of experimentation that can be applied to various applications and tested against data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.