Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
复制标题
DOI：
--
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Niru Maheswaranathan;Alex H. Williams;Matthew D. Golub;S. Ganguli;David Sussillo
中科院分区：
文献类型：
--
作者：
Niru Maheswaranathan;Alex H. Williams;Matthew D. Golub;S. Ganguli;David Sussillo
循环神经网络（RNNs）是一种广泛用于对序列数据进行建模的工具，但它们常常被当作难以理解的黑箱。给定一个经过训练的循环网络，我们希望对其进行逆向工程——获得关于它如何解决特定任务的定量、可解释的描述。即使对于简单的任务，对循环网络如何工作的详细理解，或者如何形成这种理解的方法，仍然难以捉摸。在这项工作中，我们使用动力系统分析的工具对经过训练用于执行情感分类（一项基础的自然语言处理任务）的循环网络进行逆向工程。给定一个经过训练的网络，我们找到循环动力学的不动点，并在这些不动点周围将非线性系统线性化。尽管它们在理论上有能力执行复杂的高维计算，但我们发现经过训练的网络收敛到高度可解释的低维表示。特别是，不动点的拓扑结构和相应的线性化动力学揭示了RNN内的一种近似的线吸引子，我们可以利用它来定量地理解RNN如何解决情感分析任务。最后，我们发现这种机制存在于在多个数据集上训练的各种RNN架构（包括LSTM、GRU和普通RNN）中，这表明我们的发现并非特定于某一种架构或数据集。总体而言，这些结果表明，在一系列循环网络中可能会出现令人惊讶的通用且人类可解释的计算。
Recurrent neural networks (RNNs) are a widely used tool for modeling sequential data, yet they are often treated as inscrutable black boxes. Given a trained recurrent network, we would like to reverse engineer it-to obtain a quantitative, interpretable description of how it solves a particular task. Even for simple tasks, a detailed understanding of how recurrent networks work, or a prescription for how to develop such an understanding, remains elusive. In this work, we use tools from dynamical systems analysis to reverse engineer recurrent networks trained to perform sentiment classification, a foundational natural language processing task. Given a trained network, we find fixed points of the recurrent dynamics and linearize the nonlinear system around these fixed points. Despite their theoretical capacity to implement complex, high-dimensional computations, we find that trained networks converge to highly interpretable, low-dimensional representations. In particular, the topological structure of the fixed points and corresponding linearized dynamics reveal an approximate line attractor within the RNN, which we can use to quantitatively understand how the RNN solves the sentiment analysis task. Finally, we find this mechanism present across RNN architectures (including LSTMs, GRUs, and vanilla RNNs) trained on multiple datasets, suggesting that our findings are not unique to a particular architecture or dataset. Overall, these results demonstrate that surprisingly universal and human interpretable computations can arise across a range of recurrent networks.