Mathematical Sciences: Existence and Computation of Optimal Markov Controls for Adaptive Control Problems

点击翻译按钮获取中文摘要

9404990 Helmes/Stockbridge This project is supported jointly by the Applied Mathematics Program and the Statistics and Probability Program. It goal is to provide a constructive approach to the existence and identification of Markov controls for adaptive and singular control problems. The mathematical tools developed extend and provide a new approach to the solution of stochastic control problems. The focus will be on linear programming (LP) methods which arise naturally when the objective is to optimize a long-term average criterion. These LP methods will be extended to finite horizon and infinite horizon discounted criteria. A direct characterization of the stationary distributions identifies Markov controls and allows the value to be obtained as the solution of an infinite-dimensional LP rather than as a solution to the Hamilton-Jacobi-Bellman partial differential equation. The form of the constraints leads to a natural approximation procedure which allows numerical computation of nearly optimal controls. This project is supported jointly by the Applied Mathematics Program and the Statistics and Probability Program. Its goal is the development of a new, constructive approach to the solution of adaptive and singular stochastic control problems and of approximation procedures for the numerical computation of nearly optimal controls. This research is expected to have many important applications. Two such example are the task of managing a portfolio of stocks so as to optimize some financial goal, and the problem of valuing options. The theory of option pricing is well-understood when there are no transaction costs, but existing methods are no longer appropriate when costs are part of the transaction. Other examples of adaptive control applications range from robotic manipulators to flight contpol systems of high-performance aircraft. All these systems have to operate over a wide range of working conditions, leading to great variability in the system paramete rs. The complexity of these systems requires the implementation of adaptive controllers. The practical implementation of the controls further requires sophisticated numerical methods to find nearly optimal adaptive controllers.