CAREER: Value Function Approximation for Control of Complex Systems

9985229Van roy2 .本研究致力于发展简化可靠的值函数近似计算方法。一个成功的结果将是在复杂系统的控制中广泛使用和有效的近似算法。提出的近似方法建立在神经动态规划领域的工作基础上，该领域有时被称为“近似动态规划”或“强化学习”。将开发的算法是基于近似值迭代，时间差，学习和线性规划。一种涉及使用与简化问题相关的价值函数的“特征选择”方法也将被探讨。为了促进对正在开发的方法的实用主义观点，并为评估想法提供一个测试平台，选择了两个应用程序来在项目中发挥不可或缺的作用：动态风险管理和多类排队网络的控制。这个项目的教育部分包括一门新的研究生水平的神经动力学规划课程，以及对当前课程的重新调整，以更加强调计算，培养对系统变得更复杂时使用近似的欣赏，并促进跨许多学科的随机控制问题的统一观点

9985229Van RoyThis proposed research is devoted to the development of streamlined and reliable computational methods for value function approximation. A successful outcome would be approximation algorithms that are widely-accessible and effective in the control of complex systems.Proposed approximation methods build on work in the area of neuro-dynamic programming which is sometimes called "Approximate Dynamic Programming" or "Reinforcement Learning." Algorithms that will be developed are based on approximate value iteration, temporal-difference, learning, and linear programming. A method for "feature selection" involving the use of value functions associated with simplified problems will also be explored.To promote a pragmatic view of methods under development, and to provide a testbed for evaluation of ideas, two applications have been chosen to play integral roles in the project: dynamic risk management and the control of multiclass queuing networks.The educational component of this project includes a new graduate level course on neurodynamic programming together with a realignment of current courses to incorporate a greater emphasis on computation, to foster an appreciation for the use of approximations when system become more complex, and to promote a unified view of stochastic control problems across many disciplines.***