Collaborative Research: Theoretical and Algorithmic Advances in Sequential Adaptive Decisions

许多现代操作决策可以被建模为数据驱动的优化问题，其中控制器必须在完全不知道模型的某些参数的情况下运行。基本情况涉及一个控制器，他必须从一组潜在不同的操作过程中反复选择一个操作，从而产生一个结果。例如，在适应性临床试验中，受试者必须按顺序分配到治疗组或对照组，结果是受试者的健康状况。在这个问题的背景下，控制者希望根据揭示的结果顺序有效地在“发现”(探索)和“改进”(利用)之间进行管理。这样的战略有可能更有效地收集信息，并可能导致改进决策。该奖项将支持放宽现有模型的许多限制的基础研究。该项目的成果在许多不同的领域都有应用，包括数据驱动的运营管理、在线学习和优化、医疗保健和自适应路由。该奖项将支持有才华的研究生参与这项研究，PI将创建级别课程模块，将这些结果整合到关于随机过程的课程中。该框架的传统假设包括来自独立总体的独立和相同分布的样本，只有在激活时才能了解试验结果，以及无限的计算资源。这个项目的主要目标是向以下方向扩展知识的前沿：1)考虑受某些已知关系或性质限制的模型，以便来自一个行动过程的数据可能是关于任何或所有其他行动过程的信息；2)调查相对于更一般的目标(如中位数、分位数或变异性)进行优化的模型；3)将基本理论扩展到在每个阶段已知以预定方式修改奖励价值的“背景”的模型；以及4)考虑每个行动过程的奖励分配可能随时间而改变的模型。这项研究有望产生新的理论结果、高效的算法和分析工具，以便以最佳或接近最佳的方式收集和利用数据。

A wide range of modern operational decisions can be modeled as data driven optimization problems where a controller must act without full knowledge of certain parameters of a model. The basic situation involves a controller who must repeatedly choose an action, resulting in an outcome, from an existing set of potentially different courses of actions. For instance, in adaptive clinical trials, subjects must be sequentially assigned to a treatment or control group, with the outcome being the health of the subject. In this problem context, the controller would like to efficiently manage between "discovery" (exploration) and "refinement" (exploitation) based on the revealed sequence of outcomes. Such a strategy has the potential to gather information more efficiently and may lead to improved decision making. This award will support fundamental research that relaxes many of the restrictions of existing models. The results of this project have applications in many different areas, including data driven operations management, online learning and optimization, health care, and adaptive routing. The award will support the participation of talented graduate students in this research, and the PIs will create level course modules to integrate these results in courses on stochastic processes. Traditional assumptions for this framework include independent and identically distributed samples from independent populations, learning the outcome of a trial only upon activation, and unlimited computational resources. The primary goal of this project is to extend the frontiers of knowledge in the following directions: 1) consideration of models constrained by some known relation or property, so that data from one course of actions is potentially informative about any or all other courses of action; 2) investigation of models where the optimization is with respect to more general objectives such as the median, the quantile, or the variability; 3) extension of the underlying theory to models where at each stage a "context" is made known that modifies the value of the reward in a predetermined fashion; and 4) consideration of models where the distribution of rewards for each course of action may change over time. The research is expected to lead to new theoretical results, efficient algorithms, and analytical tools for collecting and utilizing data in optimal or near optimal ways.