A Qualitative Study of Time-Lagged Recurrent Networks

时滞递归网络（TLRN）在非线性系统识别和控制方面比前馈多层神经网络更强大。该项目将在两个领域研究TLRN的几个基本重要的理论和实践问题：结构自适应性和鲁棒性分析：对于使用神经网络的非线性函数逼近，将研究时间滞后循环网络的行为，并与前馈神经网络进行比较。我们将会看到，由于网络中递归的存在，时滞递归网络将表现出与时变参数的多层前馈神经网络行为等效。期望具有时变参数的多层前馈神经网络具有自适应行为；研究表明，具有固定参数的时滞递归网络通常会表现出自适应行为。对时滞递归网络在参数扰动下的表现也将进行深入的研究。结果将作为允许参数扰动的边界给出，保证保持网络的期望性能。基于能量函数方法的训练：递归神经网络的稳定性，它可以转化为网络中的收敛行为和有界信号和参数，这是一个基本的兴趣。时间滞后递归网络的稳定性将使用Lyapunov的第二种方法进行研究。在应用李雅普诺夫第二方法时，需要构造一个李雅普诺夫函数（或能量函数）。稳定性是由Lyapunov函数是正定的并且（单调地）随时间递减的特性保证的。这一稳定性分析结果将为神经网络系统辨识训练提供新的思路，从而得到保证收敛于全局最小值或解的训练算法。

ECS-9732785LiuTime-Lagged Recurrent Networks (TLRN)are known to be more powerful than feedforward multilayer neural networks and in nonlinear systems identification and control. This project will investigate several fundamentally important theoretical and practical questions concerning TLRN's in two areas:Structural adaptivity and robustness analysis: For nonlinear functionapproximation using neural networks, the behavior of time-laggedrecurrent networks will be studied in comparison to feedforward neural networks. It will be shown that, due to the existence of recurrencies in the network,time-lagged recurrent networks will exhibit behavior equivalent tomultilayer feedforward neural networks with time-varying parameters. It isexpected that a multilayer feedforward neural network with time-varyingpaxameters will exhibit adaptive behavior; and the study will show that time-lagged recurrent networks with fixed parameters will generally exhibit adaptivebehavior. A thorough study of how time-lagged recurrent networks willperform under parameter perturbations will also be conducted. Resultswill be given as bounds for permissible parameter perturbations whichguarantee to retain the desired performance of the network.Training based on energy function approach: Stability of recurrentNeural networks which can be translated into convergent behavior and boundedsignals and parameters in the network is of fundamental interest. Thestability properties of tiine-lagged recurrent networks will be studiedusing the Lyapunov's second method. In applying the Lyapunov second method, oneneeds to construct a Lyapunov function (or energy function). Stability isguaranteed by the fart that the Lyapunov function is positive definite andis (monotonically) decreasing over time. This stability analysis result willshed new lights on the training of neural networks for systems identification,which will lead to training algorithms with guaranteed convergence to the globalminimum or to a solution.