Recurrent neural networks for stochastic control problems with delay
Recurrent neural networks for stochastic control problems with delay
复制标题
DOI：
10.1007/s00498-021-00300-3
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Jiequn Han;Ruimeng Hu
中科院分区：
文献类型：
--
作者：
Jiequn Han;Ruimeng Hu
时滞随机控制问题是具有挑战性的，由于系统的路径依赖性，因此其固有的高维。在本文中，我们提出并系统地研究了基于深度神经网络的算法来解决具有延迟特征的随机控制问题。具体来说，我们使用神经网络进行序列建模（例如，循环神经网络，例如长短期记忆）来参数化策略并优化目标函数。所提出的算法进行了测试，在三个基准的例子：一个线性二次问题，固定有限延迟的最优消费，投资组合优化与完整的内存。特别是，我们注意到，与前馈网络相比，递归神经网络的架构自然地捕获了具有更大灵活性的路径依赖特征，并通过更有效和更稳定的网络训练产生更好的性能。这种优越性甚至在具有无限时滞的完全记忆的投资组合优化的情况下也是显而易见的。
Stochastic control problems with delay are challenging due to the path-dependent feature of the system and thus its intrinsic high dimensions. In this paper, we propose and systematically study deep neural network-based algorithms to solve stochastic control problems with delay features. Specifically, we employ neural networks for sequence modeling (e.g., recurrent neural networks such as long short-term memory) to parameterize the policy and optimize the objective function. The proposed algorithms are tested on three benchmark examples: a linear-quadratic problem, optimal consumption with fixed finite delay, and portfolio optimization with complete memory. Particularly, we notice that the architecture of recurrent neural networks naturally captures the path-dependent feature with much flexibility and yields better performance with more efficient and stable training of the network compared to feedforward networks. The superiority is even evident in the case of portfolio optimization with complete memory, which features infinite delay.