Adaptive Critics for Nonlinear Continuous-Time Systems

智力优势：PI建议研究自适应动态规划（ADP）的连续时间版本，这是一种新兴的新型设计，它为受随机干扰、非线性和需要预见的大型复杂系统的（近似）最佳管理提供了类似大脑的能力的希望。他将一起研究稳定性证明、收敛分析和新设计。具体的研究目标包括用于输出反馈和非线性滤波的神经网络ADP，具有熵和非标准值函数的神经控制，基于模型的连续时间ADP和用于无模型连续时间ADP的小时间步长近似整定。神经网络提供了一种近似未知非线性函数的方法，需要作为一般ADP的一部分，对于有限数量的参数，它比传统的近似器具有更高的精度。更广泛的影响：PI计划编写一本关于ADP在工程中的教科书，这是目前非常需要的。目前可用的相关书籍要么是普通学生难以接触到的论文集（如《学习和近似动态规划手册》，IEEE出版社，2004年），要么是专注于为心理学或人工智能开发的强化学习方法的教科书，这些方法不能很好地扩展到大型工程任务。PI还将把新的设计/算法纳入自动化和机器人研究所（ARRI）的实际应用中，纳入本科课程，以及他为高中学生设计和学习智能控制系统的课程。

Intellectual Merit: The PI proposes to work on continuous-time versions of Adaptive Dynamic Programming (ADP), an emerging novel class of designs which offers hope of brain-like capabilities in the (approximately) optimal management of large complex systems subject to random disturbance, nonlinearity and a need for foresight. He will work on stability proofs, convergence analysis and new designs, together. Specific research goals involve neural network ADP for output feedback and nonlinear filtering, neurocontrol with entropy and nonstandard value functionals, model-based continuous time ADP and small-time-step approximate tuning for model-free continuous time ADP.Neural networks provide a way to approximate unknown nonlinear functions, needed as part of general ADP, with greater accuracy than traditional approximators for a limited number of parameters.Broader Impacts: The PI plans to write a textbook on ADP in engineering, which is needed very greatly at the present time. The related books currently available are either collections of papers less accessible to the average student (such as the Handbook of Learning and Approximate Dynamic Programming, IEEE Press, 2004) or textbooks which focus on methods of reinforcement learning developed for psychology or artificial intelligence, which do not scale well to large engineering tasks. The PI will also insert the new designs/algorithms into the practical applications worked on at the Automation and Robotics Research Institute (ARRI), into undergraduate courses, and into his program for high school students in Hands-On Design and Learning for Intelligent Control Systems.