Nerve Theorems for Fixed Points of Neural Networks
Nerve Theorems for Fixed Points of Neural Networks
复制标题
神经网络不动点的神经定理
DOI：
10.1007/978-3-030-95519-9_6
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Curto, C.
中科院分区：
文献类型：
--
作者：
Santander, D. E.;Ebli, S.;Patania, A.;Sanderson, N.;Burtscher, F.;Morrison, K.;Curto, C.
众所周知，非线性网络动力学很难理解。在这里，我们研究了一类递归神经网络称为组合阈值线性网络（CTLN），其动力学是由一个有向图的结构。它们是TLN的一个特例，TLN是计算神经科学中建模神经活动的流行框架。在先前的工作中，CTLN被发现在数学上令人惊讶地易于处理。对于小型网络，网络动力学的不动点通常可以通过一系列可以直接应用于底层图的图规则来完全确定。对于更大的网络，理解网络的全局结构如何与局部属性相互作用仍然是一个挑战。在这项工作中，我们提出了一种方法，覆盖图的CTLN与一组更小的有向graphsthat反映了当地的活动流。虽然方向图可能具有或可能不具有前馈架构，但它们的固定点结构指示前馈动态。图覆盖的组合结构由覆盖的神经来捕捉。神经是一个更小、更简单的图形，更适合图形分析。我们提出了三个神经定理，从神经的结构对底层网络的不动点提供了强有力的约束。然后，我们用一些例子来说明这些定理的力量。值得注意的是，我们发现，神经不仅约束CTLN的不动点，而且还提供了洞察的瞬态和渐近动力学。这是因为网络中的活动流倾向于沿着神经的边缘流动。
Nonlinear network dynamics are notoriously difficult to understand. Here we study a class of recurrent neural networks called combinatorial threshold-linear networks (CTLNs) whose dynamics are determined by the structure of a directed graph. They are a special case of TLNs, a popular framework for modeling neural activity in computational neuroscience. In prior work, CTLNs were found to be surprisingly tractable mathematically. For small networks, the fixed points of the network dynamics can often be completely determined via a series ofgraph rulesthat can be applied directly to the underlying graph. For larger networks, it remains a challenge to understand how the global structure of the network interacts with local properties. In this work, we propose a method of covering graphs of CTLNs with a set of smallerdirectional graphsthat reflect the local flow of activity. While directional graphs may or may not have a feedforward architecture, their fixed point structure is indicative of feedforward dynamics. The combinatorial structure of the graph cover is captured by thenerveof the cover. The nerve is a smaller, simpler graph that is more amenable to graphical analysis. We present three nerve theorems that provide strong constraints on the fixed points of the underlying network from the structure of the nerve. We then illustrate the power of these theorems with some examples. Remarkably, we find that the nerve not only constrains the fixed points of CTLNs, but also gives insight into the transient and asymptotic dynamics. This is because the flow of activity in the network tends to follow the edges of the nerve.