Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
复制标题
DOI：
--
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
Hendrik Strobelt;Sebastian Gehrmann;Bernd Huber;H. Pfister;Alexander M. Rush
中科院分区：
文献类型：
--
作者：
Hendrik Strobelt;Sebastian Gehrmann;Bernd Huber;H. Pfister;Alexander M. Rush
递归神经网络，特别是长短期记忆网络（LSTM），是一种非常有效的序列建模工具，可以学习其序列输入的密集黑盒隐藏表示。有兴趣更好地理解这些模型的研究人员研究了隐藏状态表示随时间的变化，并注意到一些可解释的模式，但也有显着的噪音。在这项工作中，我们提出LSTMV是一个可视化的分析工具，用于递归神经网络，重点是了解这些隐藏的状态动态。该工具允许用户选择假设输入范围以关注局部状态变化，将这些状态变化与大型数据集中的类似模式相匹配，并将这些结果与特定于域的结构注释对齐。我们进一步展示了该工具的几个用例，用于分析包含嵌套，短语结构和和弦进行的数据集上的特定隐藏状态属性，并演示了该工具如何用于隔离模式以进行进一步的统计分析。
Recurrent neural networks, and in particular long short-term memory networks (LSTMs), are a remarkably effective tool for sequence modeling that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise. In this work, we present LSTMVis a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows a user to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with domain specific structural annotations. We further show several use cases of the tool for analyzing specific hidden state properties on datasets containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis.