Stochastic Control Problems and Numerical Methods

马尔可夫链近似数值方法是目前连续时间随机控制问题的首选方法。这个项目将为具有时滞的非线性随机系统开发类似的算法。计算和仿真表明，考虑时延的适当控制可以显著提高系统性能。然而，目前关于非线性问题的算法或收敛证明的资料很少。由于动力学中的时滞，原始模型问题是无限维的，数值算法的内存需求是一个严重的问题，并给算法的结构和收敛证明带来了重大的技术挑战。由于时滞方程对模型摄动非常敏感，因此必须小心。正在开发对原始模型的近似，以简化数值算法的记忆问题，并正在开发收敛证明。证明方法是纯概率的，基于弱收敛技巧，因此非常灵活和通用。此外，数值算法可以表示为马尔可夫链模型的控制问题，这使得该过程具有直观的意义。通信系统是长期关注的，因此遍历(或每单位时间的平均成本)成本标准很受欢迎。对于时滞情况，几乎没有相关的遍历理论，从分析和算法的角度来看，有效的数值逼近问题几乎是完全开放的。将扩展到奇异和脉冲控制以及中立型方程。具有时滞的非线性受控随机动态模型是普遍存在的。它们出现在互联网控制的模型中(以及通信、控制、生物学、金融经济学等领域的其他领域)。其中控制信号受到通信延迟的影响。事实上，通信延迟的影响是设计有效和稳定使用的互联网协议的主要考虑因素。主要研究人员S数值求解随机控制问题的方法被广泛应用于(控制系统、经济、金融模型)无时滞连续时间问题。这项工作将把方法的范围扩展到一般的非线性随机时滞系统。目前，人们对此类问题知之甚少。将要开发的算法和分析方法将使这类系统能够更有效地控制，并展示在当前重要的具体问题上的有用性。

The Markov chain approximation numerical methods are the ones of current choice for continuous time stochastic control problems. This project will develop analogous algorithms for nonlinear stochastic systems subject to delays. Computations and simulations show that appropriate controls that take the delays into account can improve performance considerably. Yet little is currently available concerning algorithms or convergence proofs for nonlinear problems. Due to the delays in the dynamics, the original model problem is infinite-dimensional, and the memory requirements of the numerical algorithms are a serious concern and cause major technical challenges for both the structure of the algorithms and the proofs of convergence. One must be careful since the delay equations can be very sensitive to model perturbations. Approximations to the original models are being developed to simplify the memory problems for the numerical algorithms, and convergence proofs are being developed. The methods of proof are purely probabilistic, being based on weak convergence techniques, hence are very flexible and general. Furthermore the numerical algorithms can be represented as control problems for Markov chain models, which gives the procedure intuitive meaning. Communications systems are of interest for a long time period, hence the popularity of the ergodic (or average cost per unit time) cost criterion. There is little relevant ergodic theory for the delay case and the problem of effective numerical approximations is almost completely open, from both the analytical and algorithmic perspectives. Extensions to singular and impulsive controls and neutral equations will be carried out.Nonlinear controlled stochastic dynamical models subject to delays are ubiquitous. They occur in models of internet control (and elsewhere in communications, control, biology, financial economics, etc.) where the control signals are subject to communications delays. Indeed, the effects of communications delays are major concerns in the design of internet protocols for efficient and stable usage. The Principal Investigator?s methods for numerically solving stochastic control problems are widely used (in control systems, economics, financial models) for continuous time problems without delays. This work will extend the range of the methods to general nonlinear stochastic systems with delays. Little is known about such problems at present. The algorithms and analytical methods to be developed will enable much more effective control of such systems and demonstrate the usefulness on concrete problems of current importance.