Multi-armed Bandit Problems with Covariates

具有协变量的多臂老虎机问题

• 批准号: 1106576
• 负责人: Yuhong Yang
• 金额: $ 25万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-06-01 至 2014-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1106576&HistoricalAwards=false
• 关键词: Multi; armed; Bandit; Problems; Covariates

Multi-armed bandit (MAB) refers to a class of sequential decision making problems where in each step one needs to choose a population from which a random reward will be generated. The goal is to maximize the total accumulated reward. The literature on MAB, with few exceptions, ignores available covariates. In this project, the PI will study MAB with covariates in general frameworks and develop methodologies as well as theories for various applications. The project will 1) provide methods for selecting key covariates; 2) establish consistency in variable selection; 3) establish consistency of the allocation rule in terms of the accumulated reward; 4) derive the rate of convergence of the accumulated reward relative to the oracle choices. In addition, nonparametric estimation of the mean reward functions and model combinations will be utilized for achieving higher expected reward. Strategies that simultaneously achieve high expected reward and also provide sufficient information for identifying the best arm (with high probability) will be sought.In practice of medicine, treatments previously shown to be the best at population levels in clinical trials are given to new patients with minimal consideration of his/her own personal characteristics such as genetic profile. If practically feasible, there is every reason for a patient to be treated in a way that the outcomes of all previous treatments of patients with the same disease will have been taken into account and consequently the most promising individualized treatment is selected based on genetic information, clinical assessments, and all the accumulated trial/treatment results. The proposed research will set up statistical frameworks and build theories and methodologies for application of individualized medicine using the statistical machinery of sequential allocation with covariates. Besides medicine, sequential allocation has applications in operations research, industrial engineering, economics and other fields. Due to the ease of getting and processing information furnished by the exponential growth of modern technology, with new research to bring effective use of key predictors, applications of sequential allocation with covariates will make a real impact, saving lives, improving health, promoting business, and reducing operating cost for the society.

多臂盗贼(MAB)指的是一类序贯决策问题，其中每一步都需要选择一个种群，并从中产生随机报酬。目标是最大化累积的总回报。除了极少数例外，有关MAB的文献忽略了可用的协变量。在这个项目中，PI将在总体框架中研究带有协变量的人与生物圈，并为各种应用开发方法和理论。该项目将1)提供选择关键协变量的方法；2)建立变量选择的一致性；3)根据累积奖励建立分配规则的一致性；4)推导累积奖励相对于先知选择的收敛速度。此外，将利用平均报酬函数和模型组合的非参数估计来获得更高的期望报酬。在医学实践中，以前在临床试验中被证明是人群水平上最好的治疗方法被给予新患者，而最少考虑他/她自己的个人特征，如基因特征。如果实际可行，患者完全有理由接受这样的治疗，即考虑到相同疾病患者以前所有治疗的结果，因此根据遗传信息、临床评估和所有累积的试验/治疗结果选择最有希望的个体化治疗。建议的研究将建立统计框架，并利用协变量序贯分配的统计机制建立个体化药物应用的理论和方法。除医学外，序列分配在运筹学、工业工程、经济学等领域都有应用。由于现代技术的指数级增长提供的信息易于获取和处理，随着新的研究带来关键预测因素的有效使用，带协变量的顺序分配应用将产生真正的影响，拯救生命、改善健康、促进商业发展，并为社会降低运营成本。