A Unified Multi-Stage Approach to Generalized Sequential Decision Making Problems with Covariates

顺序决策问题是现代重要应用中经常遇到的优化任务。随着数据驱动技术的快速发展，各种各样的应用实例包括智能手机用户的在线服务推荐、医疗服务干预计划的智能实施、金融服务自动化处理等。通常，面对多个决策臂，服务提供者需要为每个即将到来的服务用户选择一个要交付的决策臂，并以最大化所有这些服务用户的总体回报和利益为目标。此外，在这个大数据时代，服务提供商通常可以访问个人用户协变量和指标，这在个性化（移动、医疗或商业）服务决策中具有很大的前景，可以提高用户奖励结果。该项目将显著推进协变量序列决策问题的方法和理论，并解决多个统计相关领域（如计算机科学、运筹学、商业分析、健康科学和更广泛的机器学习社区）感兴趣的重要问题。将通过与商业和医学研究界的密切跨学科合作，促进个性化服务的前景。该奖学金资助的研究生将学习统计理论、编程和数据分析。在参数和非参数框架下，将发展一种统一的多阶段方法来最优解决一系列广义序列决策问题，这些问题被表述为带有协变量的多臂随机强盗问题。具体而言，研究人员的目标是(1)开发一种新的算法，在与现有工作大大放松的假设下处理高维用户协变量，同时提高性能；结合了一类高维回归方法和非i.d的新技术工具。从样本中继承算法，建立严格的有限时间后悔分析和有用的统计特性；(2)提出了一种新的非参数框架作为带协变量的删减盗匪问题，并在可能删减的奖励响应和非线性决策边界下表现出最优累积遗憾和灵活使用；(3)研究了一类高维降维方法，以缓解非参数回归设置中的维数问题，并显著扩展了经典非参数方法在高维问题中的应用。从商业和医学研究问题中获取决策灵感的复杂协变量结构的理论和实证研究将为研究生和本科生创造宝贵的培训和研究机会。该奖学金资助的研究生将学习统计理论、编程和数据分析。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Sequential decision making problems are commonly encountered optimization tasks with important modern applications. With rapid advances in data-driven technology, the diverse application examples include online service recommendation for smart phone users, intelligent implementation of intervention plans for medical service, automated financial service processing, and many others. Generally, faced with multiple decision arms, a service provider needs to choose one to be delivered for each upcoming service user, and targets to maximize the overall reward and benefits for all these service users. Furthermore, in this Big Data era, individual user covariates and metrics are often accessible to service providers, which holds great promise in personalized (mobile, medical, or business) service decision making to enhance user reward outcomes. This project will significantly advance the methods and theory for the sequential decision making problems with covariates, and address important questions that are also of interests to multiple statistics-related fields such as computer science, operations research, business analytics, health sciences, and broader machine learning communities. The promising use of personalized service will be promoted through close interdisciplinary collaborations with business and medical research communities. The graduate student supported by this grant will help with statistical theory, programming, and data analytics. Under both parametric and nonparametric frameworks, a unified multi-stage approach will be developed to optimally solve a series of generalized sequential decision making problem settings formulated as multi-armed stochastic bandit problems with covariates. In particular, the investigators aim to (1) develop a new algorithm to handle high-dimensional user covariates under assumptions much relaxed from existing work while improving performance; with integration of a class of high-dimensional regression methods and new technical tools for non-i.i.d. samples inherited from the algorithm, establish rigorous finite-time regret analysis and useful statistical properties; (2) propose a new nonparametric framework as the censored bandit problem with covariates and show optimal cumulative regret and flexible use with possibly censored reward response and non-linear decision boundary; (3) study a class of high-dimensional dimension reduction methods to mitigate curse of dimensionality issues in the nonparametric regression settings and significantly extend the use of classical nonparametric methods in high dimensional problems. Both theoretical and empirical studies to incorporate complex covariate structures inspired from business and medical research questions for decision making will create valuable training and research opportunities for graduate and undergraduate students. The graduate student supported by this grant will help with statistical theory, programming and data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.