Learning Optimal Control Using Forward Backward Stochastic Differential Equations

随机系统是那些行为是随机的，不能准确预测，但可以统计分析的系统。随机最优控制在机器人学、空间探索、自治系统、金融、计算神经科学和计算生物学等领域有着广泛的应用。尽管随机最优控制理论有很长的历史，但现有的方法在动态结构、成本函数的形式以及控制策略和随机扰动之间的联系方面存在局限性。这些假设限制了这种控制方法的适用性，只适用于通常具有更简单描述的特殊类问题。该项目将把随机最优控制的适用范围扩展到更广泛的随机优化问题。该项目的教育效益包括为高级本科生和研究生开发一门新课程和教材。具体地说，本研究旨在利用正倒向随机微分方程的理论及其与倒向非线性偏微分方程解的概率表示之间的联系来开发新的、可扩展的随机控制算法。为了帮助今后在这些领域研究和采用这项工作，在该项目过程中开发的代码、数据和结果将免费分发给科学界。PIS计划集成自适应重要性采样和正反向随机微分方程的强大方法，以开发可扩展的迭代随机控制算法。此外，本研究计划将正倒向随机微分方程的理论推广和推广到随机微分对策、控制约束和Bang-bang随机控制以及非光滑代价函数下的随机控制等问题。对这些推广的工作涉及算法的发展，这将进一步将随机最优控制的适用性扩展到新的领域和新的任务。这个研究项目的教育计划有几个目标，旨在让本科生和研究生参与研究，并激励学生在随机控制和统计学的交叉点上解决具有挑战性的问题。教育方面的好处包括为高级本科生和研究生开发一门新课程和教材。

Stochastic systems are those whose behavior is random and cannot be predicted accurately but can be analyzed statistically. Stochastic optimal control has a wide range of applications in robotics, space exploration, autonomous systems, finance, computational neuroscience and computational biology. Despite the long history of stochastic optimal control theory, existing methodologies suffer from limitations related to assumptions on the structure of the dynamics, the form of cost functions, and connections between control strategies and random disturbances. These assumptions have restricted the applicability of this control method to special classes of problems that typically have simpler descriptions. This project will expand the applicability of stochastic optimal control to a broader class of stochastic optimization problems. The educational benefits of this project involve development of a new course and instructional materials for advanced undergraduate and graduate students. In particular, this research aims to develop novel and scalable stochastic control algorithms using the theory of forward-backward stochastic differential equation and their connections to probabilistic representations of solutions of backward nonlinear partial differential equations. To aid future research and adoption of this work into these domains, the code, data, and results developed during the course of this project will be distributed freely to the scientific community. The PIs plan is to integrate powerful methods on adaptive importance sampling and forward-backward stochastic differential equations to develop scalable iterative stochastic control algorithms. In addition, this research project plans to make generalizations and extensions of the theory of forward-backward stochastic differential equations to problems such as stochastic differential games, control-constrained and bang-bang stochastic control and stochastic control under non- smooth cost functions. The work on these generalizations involves the development of algorithms which, will further expand the applicability of stochastic optimal control into new domains and new tasks. The educational plan of this research project has several goals designed to engage undergraduate and graduate students in research and inspire students to work on challenging problems at the intersection of stochastic control and statistics. The educational benefits involve development of a new course and instructional materials for advanced undergraduate and graduate students.