Mathematical Sciences: Existence and Computation of Optimal Markov Controls for Adaptive Control Problems

本项目由应用数学项目和统计与概率项目联合支持。它的目标是为自适应和奇异控制问题的马尔可夫控制的存在和识别提供一种建设性的方法。所开发的数学工具扩展并提供了求解随机控制问题的新方法。重点将放在线性规划（LP）方法上，当目标是优化长期平均准则时，线性规划（LP）方法自然出现。将这些LP方法推广到有限视界和无限视界折现准则。平稳分布的直接表征识别了马尔可夫控制，并允许将值作为无限维LP的解而不是作为Hamilton-Jacobi-Bellman偏微分方程的解来获得。约束的形式导致了一个自然的近似过程，使得数值计算接近最优控制成为可能。本项目由应用数学研究方向和统计与概率研究方向联合资助。它的目标是发展一种新的、建设性的方法来解决自适应和奇异随机控制问题，并为近最优控制的数值计算提供近似程序。这项研究有望有许多重要的应用。两个这样的例子是管理股票投资组合以优化某些财务目标的任务，以及评估期权的问题。在没有交易成本的情况下，期权定价理论很好理解，但当成本是交易的一部分时，现有的方法不再适用。自适应控制应用的其他例子包括从机器人操纵器到高性能飞机的飞行控制系统。所有这些系统都必须在广泛的工作条件下运行，导致系统参数的巨大可变性。这些系统的复杂性要求实现自适应控制器。控制的实际实现进一步需要复杂的数值方法来找到接近最优的自适应控制器。

9404990 Helmes/Stockbridge This project is supported jointly by the Applied Mathematics Program and the Statistics and Probability Program. It goal is to provide a constructive approach to the existence and identification of Markov controls for adaptive and singular control problems. The mathematical tools developed extend and provide a new approach to the solution of stochastic control problems. The focus will be on linear programming (LP) methods which arise naturally when the objective is to optimize a long-term average criterion. These LP methods will be extended to finite horizon and infinite horizon discounted criteria. A direct characterization of the stationary distributions identifies Markov controls and allows the value to be obtained as the solution of an infinite-dimensional LP rather than as a solution to the Hamilton-Jacobi-Bellman partial differential equation. The form of the constraints leads to a natural approximation procedure which allows numerical computation of nearly optimal controls. This project is supported jointly by the Applied Mathematics Program and the Statistics and Probability Program. Its goal is the development of a new, constructive approach to the solution of adaptive and singular stochastic control problems and of approximation procedures for the numerical computation of nearly optimal controls. This research is expected to have many important applications. Two such example are the task of managing a portfolio of stocks so as to optimize some financial goal, and the problem of valuing options. The theory of option pricing is well-understood when there are no transaction costs, but existing methods are no longer appropriate when costs are part of the transaction. Other examples of adaptive control applications range from robotic manipulators to flight contpol systems of high-performance aircraft. All these systems have to operate over a wide range of working conditions, leading to great variability in the system paramete rs. The complexity of these systems requires the implementation of adaptive controllers. The practical implementation of the controls further requires sophisticated numerical methods to find nearly optimal adaptive controllers.