A Unified Multi-Stage Approach to Generalized Sequential Decision Making Problems with Covariates

序贯决策问题是一种具有重要应用价值的优化问题。随着数据驱动技术的快速发展，各种应用示例包括智能手机用户的在线服务推荐，医疗服务干预计划的智能实施，自动化金融服务处理等。通常，面对多个决策臂，服务提供商需要为每个即将到来的服务用户选择一个，并以最大化所有这些服务用户的整体回报和利益为目标。此外，在这个大数据时代，服务提供商通常可以访问个人用户协变量和指标，这在个性化（移动的、医疗或商业）服务决策中具有很大的前景，以提高用户奖励结果。该项目将显着推进协变量序列决策问题的方法和理论，并解决多个数学相关领域（如计算机科学，运筹学，商业分析，健康科学和更广泛的机器学习社区）感兴趣的重要问题。将通过与商业和医学研究界的密切跨学科合作，促进个性化服务的前景。该补助金支持的研究生将帮助统计理论，编程和数据分析。在参数和非参数框架下，将开发一个统一的多阶段方法，以最优地解决一系列广义序贯决策问题设置制定为多臂随机土匪问题的协变量。特别是，研究人员的目标是（1）开发一种新的算法来处理高维用户协变量的假设下，从现有的工作放松，同时提高性能;集成一类高维回归方法和新的技术工具，用于非i.i.d.（2）提出了一个新的非参数框架作为带协变量的截尾强盗问题，并给出了最优的累积后悔和灵活的使用，可能的截尾奖励响应和非线性决策边界;（3）学习一个高-维降维方法，以减轻灾难的维数问题，在非参数回归设置和显着扩展使用经典的高维问题的非参数方法。理论和实证研究，以纳入复杂的协变量结构的灵感来自商业和医学研究问题的决策将创造宝贵的培训和研究机会，为研究生和本科生。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Sequential decision making problems are commonly encountered optimization tasks with important modern applications. With rapid advances in data-driven technology, the diverse application examples include online service recommendation for smart phone users, intelligent implementation of intervention plans for medical service, automated financial service processing, and many others. Generally, faced with multiple decision arms, a service provider needs to choose one to be delivered for each upcoming service user, and targets to maximize the overall reward and benefits for all these service users. Furthermore, in this Big Data era, individual user covariates and metrics are often accessible to service providers, which holds great promise in personalized (mobile, medical, or business) service decision making to enhance user reward outcomes. This project will significantly advance the methods and theory for the sequential decision making problems with covariates, and address important questions that are also of interests to multiple statistics-related fields such as computer science, operations research, business analytics, health sciences, and broader machine learning communities. The promising use of personalized service will be promoted through close interdisciplinary collaborations with business and medical research communities. The graduate student supported by this grant will help with statistical theory, programming, and data analytics. Under both parametric and nonparametric frameworks, a unified multi-stage approach will be developed to optimally solve a series of generalized sequential decision making problem settings formulated as multi-armed stochastic bandit problems with covariates. In particular, the investigators aim to (1) develop a new algorithm to handle high-dimensional user covariates under assumptions much relaxed from existing work while improving performance; with integration of a class of high-dimensional regression methods and new technical tools for non-i.i.d. samples inherited from the algorithm, establish rigorous finite-time regret analysis and useful statistical properties; (2) propose a new nonparametric framework as the censored bandit problem with covariates and show optimal cumulative regret and flexible use with possibly censored reward response and non-linear decision boundary; (3) study a class of high-dimensional dimension reduction methods to mitigate curse of dimensionality issues in the nonparametric regression settings and significantly extend the use of classical nonparametric methods in high dimensional problems. Both theoretical and empirical studies to incorporate complex covariate structures inspired from business and medical research questions for decision making will create valuable training and research opportunities for graduate and undergraduate students. The graduate student supported by this grant will help with statistical theory, programming and data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.