On the Time Discretization of the Feynman-Kac Forward-Backward Stochastic Differential Equations for Value Function Approximation
On the Time Discretization of the Feynman-Kac Forward-Backward Stochastic Differential Equations for Value Function Approximation
复制标题
DOI：
10.1109/cdc45484.2021.9683583
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Kelsey P. Hawkins;A. Pakniyat;P. Tsiotras
中科院分区：
文献类型：
--
作者：
Kelsey P. Hawkins;A. Pakniyat;P. Tsiotras
本文提出了一种新的数值估计方法，用于值函数的费曼-卡茨表示中出现的正反向随机微分方程。与目前基于连续时间FBSDE结果离散化的数值方法相反，我们提出了一种相反的方法，首先获得策略值函数的离散时间近似，然后开发一个类似于连续时间对应的离散时间结果。该方法在函数近似阶段产生改进的数值估计器，并演示了这些值函数估计器的增强误差分析。数值结果和误差分析在标量非线性随机最优控制问题上得到了证明，并且与最先进的方法相比，它们显示了所提出的估计器在性能上的改进。
Novel numerical estimators are proposed for the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function. In contrast to the current numerical approaches based on discretization of the continuous-time FBSDE results, we propose a converse approach, by first obtaining a discrete-time approximation of the on-policy value function, and then developing a discrete-time result which resembles the continuous-time counterpart. This approach yields improved numerical estimators in the function approximation phase, and demonstrates enhanced error analysis for those value function estimators. Numerical results and error analysis are demonstrated on a scalar nonlinear stochastic optimal control problem, and they show improvements in the performance of the proposed estimators in comparison with the state-of-the-art methodologies.