Adaptive Critics for Nonlinear Continuous-Time Systems

智力优势：PI建议研究连续时间版本的自适应动态规划（ADP），这是一种新兴的新型设计，在随机干扰，非线性和前瞻性需求的大型复杂系统的（近似）优化管理中提供了类似大脑的能力。他将一起研究稳定性证明、收敛性分析和新设计。具体的研究目标包括用于输出反馈和非线性滤波的神经网络ADP，具有熵和非标准值泛函的神经控制，基于模型的连续时间ADP和无模型连续时间ADP的小时间步长近似调整。神经网络提供了一种近似未知非线性函数的方法，作为一般ADP的一部分，更广泛的影响：PI计划编写一本关于ADP工程的教科书，这是目前非常需要的。目前可用的相关书籍要么是一般学生不易获得的论文集（如学习和近似动态规划手册，IEEE出版社，2004年），要么是专注于为心理学或人工智能开发的强化学习方法的教科书，这些方法不能很好地扩展到大型工程任务。PI还将把新的设计/算法插入到自动化和机器人研究所（ARRI）的实际应用中，插入到本科课程中，并插入到他的高中生动手设计和学习智能控制系统的计划中。

Intellectual Merit: The PI proposes to work on continuous-time versions of Adaptive Dynamic Programming (ADP), an emerging novel class of designs which offers hope of brain-like capabilities in the (approximately) optimal management of large complex systems subject to random disturbance, nonlinearity and a need for foresight. He will work on stability proofs, convergence analysis and new designs, together. Specific research goals involve neural network ADP for output feedback and nonlinear filtering, neurocontrol with entropy and nonstandard value functionals, model-based continuous time ADP and small-time-step approximate tuning for model-free continuous time ADP.Neural networks provide a way to approximate unknown nonlinear functions, needed as part of general ADP, with greater accuracy than traditional approximators for a limited number of parameters.Broader Impacts: The PI plans to write a textbook on ADP in engineering, which is needed very greatly at the present time. The related books currently available are either collections of papers less accessible to the average student (such as the Handbook of Learning and Approximate Dynamic Programming, IEEE Press, 2004) or textbooks which focus on methods of reinforcement learning developed for psychology or artificial intelligence, which do not scale well to large engineering tasks. The PI will also insert the new designs/algorithms into the practical applications worked on at the Automation and Robotics Research Institute (ARRI), into undergraduate courses, and into his program for high school students in Hands-On Design and Learning for Intelligent Control Systems.