Optimal Sequential Allocation in Dynamic Environments

强盗问题已经在许多不同的背景和变化中进行了研究。绝大多数工作都集中在静态环境中，在这种环境中，每次行动产生的奖励的概率分布保持不变。这种静态模型显然无法在动态变化的环境中产生最优的决策策略。尽管这些环境在实际应用中听起来很相关，但迄今为止在统计界只得到零星的关注。所提出的研究对当前知识状态的贡献将包括为新的动态环境提出模型，这些模型是由一类重要的应用程序驱动的，设计适应动态环境的策略，分析这些策略的性能，并从有限时间（非渐近）的角度评估最优性。动态环境中的顺序分配是一个在非参数统计、机器学习和运筹学交叉领域出现的问题。该项目涉及这些领域的技术，并指出现有结果之间的基本桥梁，以形成一个更统一的主题理论。然后，该理论将作为产生具有计算效率的分配策略的基础，在临床试验、药物发现和实时网页内容优化中具有潜在的应用。

Bandit problems have been studied in many different contexts and variations. The vast majority of work focuses on a static environment, in which, at each time, the probability distribution of the reward yielded by each action remains unchanged. This static model may clearly fail to produce decision strategies that are optimal in a dynamically changing environment. Despite their sounding relevance in practical applications, such environments have received sporadic attention in the statistical community so far. The contribution of the proposed research to the current state of knowledge will consist in proposing models for new dynamic environments that are motivated by a significant class of applications, designing policies that adapt to dynamic environments, analyzing the performance of these policies and assessing optimality from a finite time (non asymptotic) point of view.Sequential allocation in dynamic environments is a problem that arises at the intersection of nonparametric statistics, machine learning and operations research. This project involves techniques from these fields and points out fundamental bridges between the extant results to form a more unified theory of the subject. This theory will then serve as a basis for producing computationally efficient allocation policies with potential applications in clinical trials, drug discovery and real time web page content optimization.