Stochastic optimal control problems with time delay: Discretisation, numerical solution, and delay sensitivity

一个最优控制问题在于控制一个系统的任务，从初始状态开始，在一定的时间段内以“最好的可能”的方式，在每个时间点选择一个控制动作，即。e.系统的某些参数的值。控制的性能是通过成本函数来衡量的。连续时间系统通常用参数化微分方程来描述。在随机情况下，微分方程具有确定性分量以及依赖于随机量的分量。时间延迟可能发生在控制或状态动态中。在前一种情况下，在控制动作的执行中或在控制动作的选择所基于的信息流中存在延迟。在后一种情况下，系统的未来演化不仅取决于当前状态（以及未来的控制和扰动），而且直接取决于系统的历史。这种控制问题的例子是由生物学或经济学的增长模型，金融市场的某些模型和工程应用。我们的项目的目的是开发，数学分析和实施程序的数值解的随机最优控制问题的时间延迟，特别是时间延迟的状态动力学。此外，我们将研究和量化控制中的小延迟的影响。长期目标是为时滞控制问题的数值处理提供理论基础和工具。

An optimal control problem consists in the task of controlling a system, starting from an initial state, over a certain period of time in a “best possible” way by choosing, at each point in time, a control action, i. e. a value for certain parameters of the system. Performance of controls is measured by means of a cost functional. A system in continuous time is usually described by a parametrised differential equation. In the stochastic case, the differential equation possesses a deterministic component as well as a component depending on random quantities. Time delay may occur in the control or in the state dynamics. In the former case, there is a delay either in the implementation of the control actions or in the information flow which underlies the choice of the control actions. In the latter case, the future evolution of the system depends not only on the current state (and on future controls and disturbances), but directly also on the history of the system. Examples of such control problems are given by growth models from biology or economics, certain models of financial markets and by applications in engineering. The aim of our project is to develop, mathematically analyse and implement procedures for the numerical solution of stochastic optimal control problems with time delay, especially with time delay in the state dynamics. In addition, we will study and quantify the effect of small delays in the control. The long-term objective is to provide a theoretical basis as well as tools for the numerical treatment of control problems with delay.