Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach
Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach
复制标题
DOI：
10.48550/arxiv.2208.03889
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Saber Jafarpour;A. Davydov;Matthew Abate;F. Bullo;S. Coogan
中科院分区：
文献类型：
--
作者：
Saber Jafarpour;A. Davydov;Matthew Abate;F. Bullo;S. Coogan
本文提出了一种基于非欧氏压缩理论的隐式神经网络训练和健壮性验证的理论和计算框架。其基本思想是将神经网络的稳健性分析归结为一个可达性问题，并使用(I)(CID：96)∞范数输入输出Lipschitz常数和(Ii)网络的紧包含函数来过逼近其可达集合。首先，对于给定的隐式神经网络，我们利用(Cid：96)∞-矩阵度量给出了其适定性的充分条件，设计了计算其fi不动点的迭代算法，并给出了其(Cid：96)∞-范数输入输出Lipschitz常数的上界。其次，我们引入了一个相关的嵌入式网络，并证明了该嵌入式网络可以用来提供原始网络可达集合的(CID：96)∞-范数盒过逼近。此外，利用嵌入式网络设计了计算原系统紧包含函数上界的迭代算法。第三，利用Lipschitz常数的上界和紧包含函数的上界，设计了两种隐式神经网络的训练和健壮性验证算法。最后，我们应用我们的算法在MNIST数据集上训练隐式神经网络，并将我们的模型的稳健性与文献中现有方法训练的模型进行了比较。
This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks based upon non-Euclidean contraction theory. The basic idea is to cast the robustness analysis of a neural network as a reachability problem and use (i) the (cid:96) ∞ -norm input-output Lipschitz constant and (ii) the tight inclusion function of the network to over-approximate its reachable sets. First, for a given implicit neural network, we use (cid:96) ∞ -matrix measures to propose sufficient conditions for its well-posedness, design an iterative algorithm to compute its fixed points, and provide upper bounds for its (cid:96) ∞ -norm input-output Lipschitz constant. Second, we introduce a related embedded network and show that the embedded network can be used to provide an (cid:96) ∞ -norm box over-approximation of the reachable sets of the original network. More-over, we use the embedded network to design an iterative algorithm for computing the upper bounds of the original system’s tight inclusion function. Third, we use the upper bounds of the Lipschitz constants and the upper bounds of the tight inclusion functions to design two algorithms for the training and robustness verification of implicit neural networks. Finally, we apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.