Attractors and computational properties of input-driven recurrent neural networks

许多理解动力系统的数学技术都局限于无输入（自治）系统。然而，对于许多应用来说，理解由输入驱动时的响应是至关重要的——这意味着我们需要理解非自治动力系统的行为。神经网络越来越流行，但尽管它们无处不在，但它们经常像“黑盒子”一样运行，这些黑盒子是根据启发式算法训练的，一旦训练完成，人们对它们的内部功能知之甚少。这对于具有内部动态状态的递归神经网络（RNNs）来说尤其如此。例如，对于训练好的RNN在给定输入时何时会出现故障，我们知之甚少，而在给定输入时，RNN可能会正常工作。这个博士项目将通过使用非自治动力系统的回拉吸引子等工具，通过检查带有输入的递归神经网络的行为来解决这些问题。该项目将建立在导师和合作者最近关于回拉吸引子、动力系统的多稳定性和rnn计算特性之间关系的工作（DOI:10.1016/j.p yd.2020.132609）的基础上。该项目旨在描述一般驱动非线性动力系统（特别是rnn）的响应特征，以及它们如何依赖于输入。这有望让我们深入了解可重复性，以及rnn的功能和故障，以及它们作为计算设备的局限性。该项目将开发一个数学框架，可以应用于门控神经网络的例子，其中不仅有连接权重的适应，而且有设置网络内时间尺度的参数的适应。

Many of the mathematical techniques for understanding dynamical systems are restricted to input-free (autonomous) systems. However, for many applications, understanding the response when driven by an input is vital - this means we need to understand the behaviour of nonautonomous dynamical system. Neural networks are increasingly prevalent but despite their ubiquity, they often operate as "black boxes" that are trained according to heuristic algorithms and little is known about their internal function once trained. This is especially the case for recurrent neural networks (RNNs) which have internal dynamical states. For example, little is known about when a trained RNN will malfunction on given an input where it might be expected to function correctly.This PhD project will approach these problem by examining the behaviour of recurrent neural networks with input, using tools such as pullback attractors from nonautonomous dynamical systems. The project will build on recent work (DOI:10.1016/j.physd.2020.132609) of the supervisor and collaborators about the relationship between pullback attractors, multistability of dynamical systems and computational properties of RNNs. The project will aim to characterize the responses of driven nonlinear dynamical systems in general (and RNNs in particular) and how they depend on inputs. This promises to give insights to repeatability, as well as function and malfunction of RNNs and their limits as computational devices. The project will develop a mathematical framework that can be applied to examples of gated neural networks where there is adaptation not only of connection weights but also of parameters that set timescales within the network.