Deep 2FBSDEs for Systems with Control Multiplicative Noise
Deep 2FBSDEs for Systems with Control Multiplicative Noise
复制标题
用于控制乘性噪声的系统的深度 2FBSDE
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Evangelos A. Theodorou
中科院分区：
文献类型：
--
作者：
Ziyi Wang;M. Pereira;Evangelos A. Theodorou
提出了一种深度递归神经网络结构，用于求解一类由完全非线性汉密尔顿雅可比-贝尔曼偏微分方程描述的随机最优控制问题.这种偏微分方程出现时，人们认为随机动力学的特点是不确定性是加性和控制乘法。具有上述特征的随机模型已被用于计算神经科学、生物学和航空航天系统，并提供比具有附加不确定性的模型更准确的致动表示。以前的文献已经建立了线性HJB理论的不足，而是依赖于非线性Feynman-Kac引理，导致二阶正向-反向随机微分方程表示。然而，所提出的解决方案，使用这种表示遭受复合误差和计算复杂性，导致缺乏可扩展性。在本文中，我们提出了一种基于深度学习的算法，该算法利用二阶前向-后向表示沿着重要性采样和基于LSTM的递归神经网络，不仅可以解决此类随机最优控制问题，还可以克服以前的方法所面临的问题，并可以很好地扩展到高维系统。对三个非线性系统的控制算法进行了测试，以证明对以前的方法的可行性和性能。
We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellman partial differential equations. Such PDEs arise when one considers stochastic dynamics characterized by uncertainties that are additive and control multiplicative. Stochastic models with the aforementioned characteristics have been used in computational neuroscience, biology and aerospace systems and provide a more accurate representation of actuation than models with additive uncertainty. Previous literature has established the inadequacy of the linear HJB theory and and instead rely on a non-linear Feynman-Kac lemma resulting in a second order forward-backward stochastic differential equations representation. However, the proposed solutions that use this representation suffer from compounding errors and computational complexity leading to lack of scalability. In this paper, we propose a deep learning based algorithm that leverages the second order Forward-Bacward SDE representation along with importance sampling and LSTM based recurrent neural networks to not only solve such Stochastic Optimal Control problems but also overcome the problems faced by previous approaches and scales well to high dimensional systems. The resulting control algorithm is tested on three non-linear systems to demonstrate feasibility and out-performance against previous methods.
影响因子：
1.5
作者：
Ioannis Exarchos;Evangelos A. Theodorou;P. Tsiotras
通讯作者：
Ioannis Exarchos;Evangelos A. Theodorou;P. Tsiotras