Learning Optimal Control Using Forward Backward Stochastic Differential Equations

随机系统是指其行为是随机的，无法准确预测，但可以进行统计分析的系统。随机最优控制在机器人、空间探索、自治系统、金融、计算神经科学和计算生物学等领域有着广泛的应用。尽管随机最优控制理论有着悠久的历史，但现有的方法受到与动态结构、成本函数形式以及控制策略与随机干扰之间的联系有关的假设的限制。这些假设限制了这种控制方法的适用性，通常有简单的描述的特殊类别的问题。本计画将随机最佳控制的适用范围扩大到更广泛的随机最佳化问题。该项目的教育效益包括为高年级本科生和研究生开发新课程和教学材料。 特别是，本研究的目的是开发新的和可扩展的随机控制算法，使用的理论，向前向后随机微分方程及其连接到向后非线性偏微分方程的解决方案的概率表示。为了帮助未来的研究和将这项工作应用到这些领域，该项目过程中开发的代码，数据和结果将免费分发给科学界。PI计划是将自适应重要性采样和前向-后向随机微分方程的强大方法相结合，开发可扩展的迭代随机控制算法。此外，本研究计划将正倒向随机微分方程理论推广和扩展到随机微分对策、控制约束和bang-bang随机控制以及非光滑代价函数下的随机控制等问题。这些概括的工作涉及算法的发展，将进一步扩大随机最优控制的适用性到新的领域和新的任务。该研究项目的教育计划有几个目标，旨在让本科生和研究生参与研究，并激励学生在随机控制和统计的交叉点上解决具有挑战性的问题。教育方面的好处包括为高年级本科生和研究生开发新课程和教材。

Stochastic systems are those whose behavior is random and cannot be predicted accurately but can be analyzed statistically. Stochastic optimal control has a wide range of applications in robotics, space exploration, autonomous systems, finance, computational neuroscience and computational biology. Despite the long history of stochastic optimal control theory, existing methodologies suffer from limitations related to assumptions on the structure of the dynamics, the form of cost functions, and connections between control strategies and random disturbances. These assumptions have restricted the applicability of this control method to special classes of problems that typically have simpler descriptions. This project will expand the applicability of stochastic optimal control to a broader class of stochastic optimization problems. The educational benefits of this project involve development of a new course and instructional materials for advanced undergraduate and graduate students. In particular, this research aims to develop novel and scalable stochastic control algorithms using the theory of forward-backward stochastic differential equation and their connections to probabilistic representations of solutions of backward nonlinear partial differential equations. To aid future research and adoption of this work into these domains, the code, data, and results developed during the course of this project will be distributed freely to the scientific community. The PIs plan is to integrate powerful methods on adaptive importance sampling and forward-backward stochastic differential equations to develop scalable iterative stochastic control algorithms. In addition, this research project plans to make generalizations and extensions of the theory of forward-backward stochastic differential equations to problems such as stochastic differential games, control-constrained and bang-bang stochastic control and stochastic control under non- smooth cost functions. The work on these generalizations involves the development of algorithms which, will further expand the applicability of stochastic optimal control into new domains and new tasks. The educational plan of this research project has several goals designed to engage undergraduate and graduate students in research and inspire students to work on challenging problems at the intersection of stochastic control and statistics. The educational benefits involve development of a new course and instructional materials for advanced undergraduate and graduate students.