Topological measurement of deep neural networks using persistent homology
Topological measurement of deep neural networks using persistent homology
复制标题
DOI：
10.1007/s10472-021-09761-3
复制
发表时间：
2021-06
影响因子：
1.2
通讯作者：
Satoru Watanabe;H. Yamana
中科院分区：
文献类型：
--
作者：
Satoru Watanabe;H. Yamana
深度神经网络（DNN）的内部表示是不可破译的，这使得调整DNN模型，控制其训练过程以及解释其输出变得困难。在本文中，我们提出了一种新的方法来研究DNN的内部表示，通过拓扑数据分析（TDA）。持久同源性（PH），TDA中的杰出方法之一，用于研究训练DNN的复杂性。我们在训练的DNN上构建团复合体，并计算DNN的一维PH。PH揭示了DNN中多个神经元在不同分辨率下的组合效应，这在不使用PH的情况下很难被捕获。使用全连接网络（FCN）以及在MNIST和CIFAR-10数据集上训练的FCN和卷积神经网络（CNN）相结合的网络进行了评估。评估结果表明，DNN的PH反映了神经元的过剩和问题的难度，使PH成为研究DNN内部表示的主要方法之一。
The inner representation of deep neural networks (DNNs) is indecipherable, which makes it difficult to tune DNN models, control their training process, and interpret their outputs. In this paper, we propose a novel approach to investigate the inner representation of DNNs through topological data analysis (TDA). Persistent homology (PH), one of the outstanding methods in TDA, was employed for investigating the complexities of trained DNNs. We constructed clique complexes on trained DNNs and calculated the one-dimensional PH of DNNs. The PH reveals the combinational effects of multiple neurons in DNNs at different resolutions, which is difficult to be captured without using PH. Evaluations were conducted using fully connected networks (FCNs) and networks combining FCNs and convolutional neural networks (CNNs) trained on the MNIST and CIFAR-10 data sets. Evaluation results demonstrate that the PH of DNNs reflects both the excess of neurons and problem difficulty, making PH one of the prominent methods for investigating the inner representation of DNNs.