Dissecting Deep Learning Networks-Visualizing Mutual Information.
Dissecting Deep Learning Networks-Visualizing Mutual Information.
复制标题
DOI：
10.3390/e20110823
复制
发表时间：
2018-10-26
期刊：
影响因子：
--
通讯作者：
Yamaguchi M
中科院分区：
文献类型：
--
作者：
Fang H;Wang V;Yamaguchi M
深度学习（DL）网络是人工智能研究中的最新革命性发展。典型的网络由多组层堆叠而成，这些层进一步由许多卷积核或神经元组成。在网络设计中，许多超参数需要在训练之前进行详细定义，以实现高交叉验证精度。然而，仅从输出层的准确性评估是不足以指定的隐藏单元在关联网络中的作用。这导致了深度学习的广泛应用与其有限的理论理解之间存在着显著的知识差距。为了缩小知识差距，我们的研究探索可视化技术来说明DL网络中的互信息（MI）。MI是一种理论测量，反映了两组随机变量之间的关系，即使它们的关系是高度非线性的，隐藏在高维数据中。我们的研究旨在了解DL单元在网络分类性能中的作用。通过对几种流行的深度学习网络的实验，表明可视化的深度学习网络的输入输出与隐含层和基本单元之间的MI及其变化模式，可以更好地理解这些深度学习单元的作用。我们对网络融合的调查提出了一种更客观的方式来潜在地评估DL网络。此外，可视化提供了一个有用的工具，以获得洞察网络性能，从而有可能促进更好的网络架构的设计，通过识别冗余和低效的网络单元。
Deep Learning (DL) networks are recent revolutionary developments in artificial intelligence research. Typical networks are stacked by groups of layers that are further composed of many convolutional kernels or neurons. In network design, many hyper-parameters need to be defined heuristically before training in order to achieve high cross-validation accuracies. However, accuracy evaluation from the output layer alone is not sufficient to specify the roles of the hidden units in associated networks. This results in a significant knowledge gap between DL’s wider applications and its limited theoretical understanding. To narrow the knowledge gap, our study explores visualization techniques to illustrate the mutual information (MI) in DL networks. The MI is a theoretical measurement, reflecting the relationship between two sets of random variables even if their relationship is highly non-linear and hidden in high-dimensional data. Our study aims to understand the roles of DL units in classification performance of the networks. Via a series of experiments using several popular DL networks, it shows that the visualization of MI and its change patterns between the input/output with the hidden layers and basic units can facilitate a better understanding of these DL units’ roles. Our investigation on network convergence suggests a more objective manner to potentially evaluate DL networks. Furthermore, the visualization provides a useful tool to gain insights into the network performance, and thus to potentially facilitate the design of better network architectures by identifying redundancy and less-effective network units.
登录
查看更多内容
DOI：
10.1109/tvcg.2016.2598831
发表时间：
2017-01-01
影响因子：
5.2
作者：
Liu, Mengchen;Shi, Jiaxin;Liu, Shixia
通讯作者：
Liu, Shixia
影响因子：
64.8
作者：
Mnih, Volodymyr;Kavukcuoglu, Koray;Hassabis, Demis
通讯作者：
Hassabis, Demis
影响因子：
14.9
作者：
Metsalu T;Vilo J
通讯作者：
Vilo J
影响因子：
8
作者：
Gu, Jiuxiang;Wang, Zhenhua;Chen, Tsuhan
通讯作者：
Chen, Tsuhan
影响因子：
2.7
作者：
Kolchinsky, Artemy;Tracey, Brendan D.
通讯作者：
Tracey, Brendan D.