A Qualitative Study of Time-Lagged Recurrent Networks

时滞递归网络（TLRN）在非线性系统辨识和控制中比前馈多层神经网络更强大。本项目将在两个方面研究有关TLRN的几个基本重要的理论和实践问题：结构自适应性和鲁棒性分析：对于使用神经网络的非线性函数逼近，将研究时滞递归网络的行为，并与前馈神经网络进行比较。由于网络中递归的存在，时滞递归网络将表现出与时变参数多层前馈神经网络等价的行为。 人们期望具有时变参数的多层前馈神经网络将表现出自适应行为;研究将表明具有固定参数的时滞递归网络通常将表现出自适应行为。 还将对时滞递归网络在参数扰动下的表现进行深入研究。 结果将给出作为允许的参数扰动的界限，保证保持所需的网络性能。基于能量函数方法的训练：递归神经网络的稳定性，可以转化为收敛行为和有界的信号和参数的网络是根本的兴趣。 本文用李雅普诺夫第二方法研究时滞递归网络的稳定性。 在应用李雅普诺夫第二方法时，需要构造一个李雅普诺夫函数（或能量函数）。 稳定性是由李雅普诺夫函数是正定的，并且随着时间的推移（单调）减少。 这一稳定性分析结果将为系统辨识的神经网络训练提供新的思路，这将导致训练算法保证收敛到全局最小值或解决方案。

ECS-9732785LiuTime-Lagged Recurrent Networks (TLRN)are known to be more powerful than feedforward multilayer neural networks and in nonlinear systems identification and control. This project will investigate several fundamentally important theoretical and practical questions concerning TLRN's in two areas:Structural adaptivity and robustness analysis: For nonlinear functionapproximation using neural networks, the behavior of time-laggedrecurrent networks will be studied in comparison to feedforward neural networks. It will be shown that, due to the existence of recurrencies in the network,time-lagged recurrent networks will exhibit behavior equivalent tomultilayer feedforward neural networks with time-varying parameters. It isexpected that a multilayer feedforward neural network with time-varyingpaxameters will exhibit adaptive behavior; and the study will show that time-lagged recurrent networks with fixed parameters will generally exhibit adaptivebehavior. A thorough study of how time-lagged recurrent networks willperform under parameter perturbations will also be conducted. Resultswill be given as bounds for permissible parameter perturbations whichguarantee to retain the desired performance of the network.Training based on energy function approach: Stability of recurrentNeural networks which can be translated into convergent behavior and boundedsignals and parameters in the network is of fundamental interest. Thestability properties of tiine-lagged recurrent networks will be studiedusing the Lyapunov's second method. In applying the Lyapunov second method, oneneeds to construct a Lyapunov function (or energy function). Stability isguaranteed by the fart that the Lyapunov function is positive definite andis (monotonically) decreasing over time. This stability analysis result willshed new lights on the training of neural networks for systems identification,which will lead to training algorithms with guaranteed convergence to the globalminimum or to a solution.