CDS&E-MSS/Collaborative Research: Sequential Design for Stochastic Control: Active Learning of Optimal Policies

这项研究项目旨在建立新的跨学科算法，融合应用概率、控制和统计建模的概念，以应对大规模优化中的计算挑战。这种新链接的创建是构建下一代高性能算法的又一步，需要这些算法来满足金融、能源储存和安全以及流行病管理等各种应用中出现的日益复杂的问题。该项目的研究议程植根于两个具体的应用领域，在这两个领域，解决工业级高保真模型至关重要。一个是随着能源基础设施向“智能电网”过渡，对循环商品资产进行高效管理，包括天然气储存、电池储存或发电厂车队。第二个是及时和有效地应对不断爆发的传染病，特别是流感。两者都面临着重大的跨学科挑战。我们看到了算法的巨大潜力，这些算法扩展了定量控制方面的能力，从而为决策者提供了更高质量的信息。我们的目标是在新一波精益随机解算器中更智能、更有针对性地使用随机数，并随后扩大利用现有计算能力可以解决的问题的规模。该项目的教育核心有助于在本科生、研究生和博士后水平上进行数学科学的跨学科培训。合作倡议还将通过两个校区(加州大学圣巴巴拉分校和芝加哥大学)以及统计学家、运筹学研究人员和工程师社区之间的思想交流，加强研究基础设施。所有算法都将被记录下来，并向更广泛的科学界公开发布。对于需要真实高保真表示的随机系统的控制，基于仿真的方案的部署仍然是关键。这个项目将为一类随机控制问题开发新的蒙特卡罗算法，通过在动态控制与序贯设计和统计学习方法之间建立新的桥梁。我们的研究议程依赖于对最优动作集的顺序、主动学习，以便算法自适应地分配计算资源，以更好地提高近似控制策略的保真度。蒙特卡罗模拟的这种有针对性的使用将近似动态编程与响应面建模联系在一起，将应用数学和统计学这两个迄今截然不同的领域结合在一起。由此产生的自适应方案将有助于节省数量级的模拟预算，扩大不确定情况下预测建模和决策的前沿。本文的研究将为海量多维状态空间的动态控制算法理论提供理论支持，在这些状态空间中，维度灾难是不可避免的。同时，统计和计算理论在这一方向上的整合将开辟跨学科定量研究的新途径。通过加强大规模控制环境中的知识发现，这些项目将促进向新环境中的实践过渡。为了接触数学、生物、物理和工程科学的不同用户，通过R包制作通用开放源码软件是该项目的主要成果，并将辅之以案例研究数据库。

This research project aims to build new cross-disciplinary algorithms that blend concepts from applied probability, control, and statistical modeling to tackle computational challenges in large-scale optimization. Creation of such new links is another step in building a next-generation of high-performance algorithms needed to meet the increasingly complex problems arising in applications as diverse as finance, energy storage and security, and the management of epidemics. The project's research agenda is grounded in two concrete application areas where it is crucial to tackle industrial-grade high-fidelity models. One is the efficient management of cycled commodity assets, including gas storage, battery storage, or fleets of power plants as energy infrastructure is transitioned to the "smart grid." A second is timely and effective response to unfolding infectious disease outbreaks, notably influenza. Both present major cross-disciplinary challenges. We see vast potential for algorithms which expand capabilities for aspects of quantitative control, and thus provide higher quality information to decision makers. Our goal is to produce a smarter, more targeted, use of random numbers in a new wave of lean stochastic solvers, and subsequently an expansion of the size of problems that can be tackled with existing computing capabilities. The educational core of the project contributes to inter-disciplinary training in mathematical sciences across undergraduate, graduate and postdoctoral levels. The collaborative initiatives will also enhance the research infrastructure through exchange of ideas between the two campuses (University of California-Santa Barbara and University of Chicago) and communities of statisticians, operations researchers and engineers. All algorithms would be documented and publicly released to the wider scientific community. Deployment of simulation based schemes remains key for control of stochastic systems that require realistic high-fidelity representations. This project will develop new Monte Carlo algorithms for a class of stochastic control problems by erecting novel bridges between dynamic control and methods of sequential design and statistical learning. Our research agenda hinges on sequential, active learning of optimal action sets, so that the algorithms adaptively allocate computing resources to better enhance fidelity of the approximated control strategies. Such targeted use of Monte Carlo simulations links approximate dynamic programming with response surface modeling, marrying two so-far disparate areas of applied mathematics and statistics. The resulting adaptive schemes will facilitate orders of magnitude savings in simulation budgets, expanding the frontier for predictive modeling and decision making under uncertainty. The proposed research will advance the theory of algorithms for dynamic control over massive multi-dimensional state spaces, where curses of dimensionality are unavoidable. Simultaneously, integration of the statistical and computational theories in this direction will open new lines of interdisciplinary quantitative research. Through enhancing knowledge discovery in large-scale control settings, the projects will facilitate transition to practice in novel contexts. With the aim of reaching out to diverse users from the mathematical, biological, physical and engineering sciences, producing general purpose open-source software via R packages is a primary deliverable of the project, and will be supplemented by a database of case studies.