Learning Deep Stochastic Optimal Control Policies Using Forward-Backward SDEs
Learning Deep Stochastic Optimal Control Policies Using Forward-Backward SDEs
复制标题
DOI：
10.15607/rss.2019.xv.070
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Ziyi Wang;M. Pereira;Evangelos A. Theodorou
中科院分区：
文献类型：
--
作者：
Ziyi Wang;M. Pereira;Evangelos A. Theodorou
在本文中，我们提出了一种新的方法，在不确定性下的决策，利用非线性随机最优控制理论，应用数学和机器学习领域的最新进展。基于某些非线性偏微分方程和正倒向随机微分方程之间的基本关系，我们开发了一个控制框架，该框架可扩展并适用于机器人和自治中的一般随机系统和决策问题配方。所提出的用于随机控制的深度神经网络架构由递归层和全连接层组成。上述算法的性能和可扩展性进行了研究，在三个非线性系统的仿真和无控制约束。最后，我们讨论了未来的发展方向及其对机器人的影响。
In this paper we propose a new methodology for decision-making under uncertainty using recent advancements in the areas of nonlinear stochastic optimal control theory, applied mathematics, and machine learning. Grounded on the fundamental relation between certain nonlinear partial differential equations and forward-backward stochastic differential equations, we develop a control framework that is scalable and applicable to general classes of stochastic systems and decision-making problem formulations in robotics and autonomy. The proposed deep neural network architectures for stochastic control consist of recurrent and fully connected layers. The performance and scalability of the aforementioned algorithm are investigated in three non-linear systems in simulation with and without control constraints. We conclude with a discussion on future directions and their implications to robotics.