Intelligent Control: A Dynamic Game Approach

该工作旨在将强化学习理论中的结果推广到与输出反馈鲁棒非线性控制相关的动态博弈问题。这主要有两个动机：(A)开发方案，以克服在设计和实现鲁棒非线性控制器时遇到的高昂的计算成本。(B)将动态博弈框架作为踏脚石，导致开发适合于姿态的分析机器，并解决智能控制问题。前者主要涉及用于逼近关键方程的离线方案，以及有效计算和表示控制策略的技术的发展。后者涉及在线方案，其中需要将识别、控制和在有限时间内提高性能的能力1与有限的计算资源相结合。后者关于系统模型和环境的可用信息较少，因此学习是该方法的重要组成部分。考虑到这些目标，需要特别强调获得表现出良好的有限时间性能的算法，并且使用有限的资源(计算)来做到这一点。此外，为了有效地集成生成的体系结构的组件，还需要为这些算法开发(有限时间)性能界限。这种方法要求首先在有限状态自动机的背景下研究问题，然后将结果推广到离散时间动态系统模型。建议的工作旨在研究：(A)扩展强化学习以获得有限时间性能界。(B)发展直接识别与控制最相关的信息(信息状态)的方案，并在有限的时间内以指定的精度做到这一点。这就要求制定权衡勘探和控制风险的措施，以便在线实施。(D)扩展当前研究强化学习的分析框架，以解释与智能相关的不可预测性。(E)利用风险敏感控制和动态博弈之间的关系来利用概率论提供的结构。(F)开发有效实现算法的体系结构和软件。从该研究项目获得的结果，加上适当的复杂性度量的开发，将导致提出和分析各种智能控制问题的框架。这样的方法将导致控制器本质上是健壮的，但能够根据系统/环境中感知的变化调整其行为。所得结果一方面可用于计算和实现鲁棒非线性控制，另一方面可用于大型复杂系统的真正自主控制。具体应用领域包括化工过程控制、半导体制造和大型通信网络控制。***

9727805BarasThe proposed work aims to extend results in reinforcement learning theory to dynamic game problems relevant to output feedback robust nonlinear control. There are two primary motivations for this:(a).To develop schemes to overcome the prohibitive computational cost encountered while designing and implementing robust nonlinear controllers.(b).Employ the dynamic game framework as a stepping stone leading to the development of an analytical machinery suitable for posing, and solving intelligent control problems.The former is concerned primarily with off-line schemes for approximating the key equations, and development of techniques to efficiently compute and represent the control policy. The latter is concerned with on-line schemes, where one needs to integrate identification, control, and the ability to improve performance in finite amount of time1 with finite computational resources. The latter has less available information on system model and environment; thus learning is an essential component of the methodology.With these objectives in mind, special emphasis needs to be placed on obtaining algorithms that exhibit good finite time performance, and do so with finite amount of resources (computational). Furthermore, in order to efficiently integrate the components of the resulting architectures, one needs to also develop (finite time) performance bounds for these algorithms. The approach calls for first studying the problem in the context of finite state automata, and then extending the results to discrete time dynamical system models. The proposed work intends to study:(a).Extensions of reinforcement learning to obtain finite time performance bounds.(b).Development of schemes to directly identify the information most relevant for control (information state), and to do so with specified accuracy in a finite amount of time. This calls for the development of measures of risk to tradeoff exploration and control for on-line implementation.(c)Model structures in. (b) that lead to reduction in complexity, and lend themselves to efficient learning.(d).Extension of the current analytical framework for studying reinforcement learning to account for the unpredictability associated with intelligence.(e).Exploiting the relationship between risk-sensitive control and dynamic games to harness the structure offered by probability theory.(f).Development of architectures, and software that efflciently implement the algorithms obtained.Results obtained from this research project, coupled with the development of appropriate complexity metrics would result in a framework for posing, and analyzing a wide variety of intelligent control problems. Such an approach would lead to controllers that are inherently robust, yet capable of adapting their behaviour to perceived changes in the system/environment. The results would be applicable to computation and implementation of robust nonlinear control at one end, to truly autonomous control for large, complex systems at the other. Specific applicatlon domains include chemical process control, semiconductor manufacturing, and control of large communication networks. ***