Stochastic optimal control problems with time delay: Discretisation, numerical solution, and delay sensitivity

最优控制问题包括控制系统的任务，从初始状态开始，在一定时间内以“最佳可能”的方式，在每个时间点选择一个控制动作，即系统的某些参数的值。控制的绩效是通过成本函数来衡量的。连续时间系统通常用参数化微分方程来描述。在随机情况下，微分方程既有确定性分量又有依赖于随机量的分量。时间延迟可能出现在控制或状态动力学中。在前一种情况下，要么在控制动作的实现中，要么在控制动作选择的基础上的信息流中存在延迟。在后一种情况下，系统的未来演化不仅依赖于当前状态（以及未来的控制和干扰），而且直接依赖于系统的历史。生物学或经济学中的增长模型、金融市场的某些模型以及工程中的应用都给出了此类控制问题的例子。我们项目的目的是开发，数学分析和实施程序的数值解决随机最优控制问题的时间延迟，特别是在状态动力学中的时间延迟。此外，我们将研究和量化小延迟在控制中的影响。长期目标是为具有延迟的控制问题的数值处理提供理论基础和工具。

An optimal control problem consists in the task of controlling a system, starting from an initial state, over a certain period of time in a “best possible” way by choosing, at each point in time, a control action, i. e. a value for certain parameters of the system. Performance of controls is measured by means of a cost functional. A system in continuous time is usually described by a parametrised differential equation. In the stochastic case, the differential equation possesses a deterministic component as well as a component depending on random quantities. Time delay may occur in the control or in the state dynamics. In the former case, there is a delay either in the implementation of the control actions or in the information flow which underlies the choice of the control actions. In the latter case, the future evolution of the system depends not only on the current state (and on future controls and disturbances), but directly also on the history of the system. Examples of such control problems are given by growth models from biology or economics, certain models of financial markets and by applications in engineering. The aim of our project is to develop, mathematically analyse and implement procedures for the numerical solution of stochastic optimal control problems with time delay, especially with time delay in the state dynamics. In addition, we will study and quantify the effect of small delays in the control. The long-term objective is to provide a theoretical basis as well as tools for the numerical treatment of control problems with delay.