Computational Intelligence: Adaptive Critics for Controller Design

这项工作的总体目标是为推进应用自适应批评（AC）方法设计神经网络控制器的艺术和实践做出实质性贡献。同时注意离线控制器和在线控制器的设计；对于潜在的工程应用来说，最重要的是在线学习的情况，其中控制器的设计在作为控制器的同时被修改。这种在线能力至少在两种情况下是有用的：1)离线设计的控制器只是近似正确的，并且控制器的设计在在线运行时得到了改进；2)即使一个高质量的控制器是离线设计的，在运行过程中，工厂或环境会发生实质性的变化，因此需要对控制器设计进行改进（因此它可以“适应”）。该研究的一个重点是从可学习性的角度来描述控制问题的各个方面——也就是说，某些情况比其他情况更容易或更容易学习；是否有一个可定义的子任务序列，使学习过程更容易实现；系统动力学如何参与学习任务的完成；等。另一个研究重点是探讨模型在控制器设计过程的各个方面所扮演的角色。这里的问题包括a)在（初始）控制器离线训练期间用于模拟工厂的模型的定性要求，以及b)在控制器和关键训练过程中用于获得所需偏导数的模型的定性要求。在研究和开发方法中，对上述问题和许多其他研究问题进行了探索，并将结果汇集在一起，以促进在线应用“学习”的自适应批评方法来完成（近似）最优控制器设计。本研究方法主要基于数学建模和对底层思想的理论化，主要基于动力系统形式化，并在高速计算机上模拟所探索的情况进行实验探索。这项研究是在波特兰州立大学新建立的计算智能实验室的背景下进行的。该实验室将过去10年跨学科系统科学博士项目的研究正式化，旨在促进其他学科研究人员之间的合作，并使研究生沉浸在这样一个跨学科的研究环境中

9904378Lendaris The overall objective of this work is to make substantive contribution to advancing the art andpractice of applying Adaptive Critic (AC) methods to the designing of neural network controllers.Attention is given to considerations for both, off-line and on line controller's design; most significant to potential engineering applications is the case of on-line learning, wherein the controller's design is modified while serving its role as a controller. Such on-line capability is usefulin at least two situations: 1) When a controller designed off-line is only approximately correct, and the controller's design is refined while it is functioning on-line; and 2) even when a high-quality controller is designed off-line to start with, substantive changes occur in the plant or environment during operation, thus requiring a refinement to the controller design (so it can "adapt"). One focus of the research is to characterize aspects of the control problem from the perspective of learnability-- that is, are certain situations more or less learnable than others; is there a definable sequence of sub-tasks that makes the learning process more likely achievable; in what way do the system dynamics enter into the accomplishment of the learning task; etc. Another research focus is to explore the roles played by models in the various aspects of the controller design process. The issues here include a) qualitative requirements on model(s) used for simulating the plant during off-line training of the (initial) controller, and b) qualitative requirements on model(s) used for obtaining the partial derivatives required during the controller and critic training procedure. The above and a number of other research questions are explored and results brought together in studying and developing methods to facilitate the on-line application of the Adaptive Critic methods of "learning" to accomplish (approximately) optimal controller design. This research methods are mostly based on mathematical modeling and theorizing of the under-lying ideas, mostly based on dynamical systems formalisms, and experimental explorations are carried out on high-speed computers that simulate the situations being explored. The research is carried out in the context of a new Computational Intelligence Laboratory beingestablished at Portland State University. This Laboratory formalizes research that has been con-ducted over the past 10 years in the interdisciplinary Systems Science Ph.D. Program, and isintended to foster increased collaboration among researchers from other disciplines and immersion of graduate students in such an interdisciplinary research context.***