Understanding Neural Networks through Dynamics

人工神经网络(ANN)最近作为模拟、预测和分类复杂输入的工具重新成为聚光灯下的焦点。我们有能力对应用程序充满信心地使用这些新的计算方法，这要求我们更好地从根本上理解为什么、如何以及在什么情况下可以依赖神经网络。在从自动驾驶汽车到警务等领域，对这种决策支持工具的依赖正在迅速增加，这是一系列明显的偏差、失败和不公正的例子，这些例子都可以追溯到基本的数学问题。最近，与递归神经网络相关的储备库计算受到了特别关注，其中大多数网络连接形成了具有固定(通常是随机选择的)连接和通过调整少量输出权重进行训练的单层递归“动态储存库”。在这个项目中，工作将集中在一类被称为回声状态网络(Echo State Networks，简称ESNs)的水库系统上。这项提议将对ESNs的结构有新的基本理解，并将开发以前未被开发的ESNs作为动力系统的特性；反过来，这将使ESNs在物理系统中的新应用成为可能，包括气候动力学。

Artificial Neural Networks (ANNs) have recently moved back into the spotlight as tools to simulate, predict, and classify complex inputs. Our ability to use these new computational methods with confidence in applications demands that we develop a better fundamental understanding of why, how, and in what situations ANNs can be relied on. Reliance on such tools for decision support is increasing rapidly in fields from autonomous vehicles to policing, as is the list of notable examples of biases, failures, and injustices that can all be traced back to fundamental mathematical issues.There has been particular focus recently on reservoir computing, related to recurrent neural networks, in which most of the network connections form a single-layer recurrent 'dynamical reservoir' with fixed (usually randomly chosen) connections and training carried out by adjusting a small number of output weights. In this project the work will focus on a class of reservoir systems known as Echo State Networks (ESNs).This proposal will develop new fundamental understanding of structure in ESNs, and will develop previously-unexploited properties of ESNs as dynamical systems; in turn this will enable new applications of ESNs to physical systems, including climate dynamics.