Reachability Analysis of Sigmoidal Neural Networks
Reachability Analysis of Sigmoidal Neural Networks
复制标题
DOI：
10.1145/3627991
复制
发表时间：
2023-10
影响因子：
2
通讯作者：
Sung-Woo Choi;Michael Ivashchenko;Luan V. Nguyen;Hoang-Dung Tran
中科院分区：
文献类型：
--
作者：
Sung-Woo Choi;Michael Ivashchenko;Luan V. Nguyen;Hoang-Dung Tran
本文扩展了星集可达性方法，以验证具有 Sigmoid 和 TanH 等 Sigmoid 激活函数的前馈神经网络 (FNN) 的鲁棒性。 Sigmoid/TanH FNN 验证中星集方法的主要缺点是由于使用线性规划求解器而导致某些情况下的可扩展性、可行性和最优性问题。我们通过提出具有符号间隔的松弛星（RStar）来克服这一挑战，这允许在 DeepPoly 中使用反向替换技术来在过度逼近激活函数时找到边界，同时保持星集的有价值的特征。 RStar 可以使用四个线性约束 (RStar4) 或两个线性约束 (RStar2) 或仅使用输出边界 (RStar0) 来过度逼近 S 型激活函数。我们在 NNV 中实现 RStar 可达性算法，并通过图像分类 DNN 基准的稳健性验证将它们与 DeepPoly 进行比较。实验结果表明，原始星形方法（即无松弛）是所有方法中最不保守但最慢的。 RStar4 的计算速度比原始星型方法快得多，并且是第二个最不保守的方法。它证明对抗对抗攻击的图像比 DeepPoly 多出 40%，平均比明星集快 51 倍。最后但并非最不重要的一点是，RStar0 是最保守的方法，对于 CIFAR10 小 Sigmoid 网络只能验证两种情况，δ = 0.014。然而，在我们的评估中，它是最快的方法，可以验证神经网络，比星集快 3528 倍，比 DeepPoly 快 46 倍。
This paper extends the star set reachability approach to verify the robustness of feed-forward neural networks (FNNs) with sigmoidal activation functions such as Sigmoid and TanH. The main drawbacks of the star set approach in Sigmoid/TanH FNN verification are scalability, feasibility, and optimality issues in some cases due to the linear programming solver usage. We overcome this challenge by proposing a relaxed star (RStar) with symbolic intervals, which allows the usage of the back-substitution technique in DeepPoly to find bounds when overapproximating activation functions while maintaining the valuable features of a star set. RStar can overapproximate a sigmoidal activation function using four linear constraints (RStar4) or two linear constraints (RStar2), or only the output bounds (RStar0). We implement our RStar reachability algorithms in NNV and compare them to DeepPoly via robustness verification of image classification DNNs benchmarks. The experimental results show that the original star approach (i.e., no relaxation) is the least conservative of all methods yet the slowest. RStar4 is computationally much faster than the original star method and is the second least conservative approach. It certifies up to 40% more images against adversarial attacks than DeepPoly and on average 51 times faster than the star set. Last but not least, RStar0 is the most conservative method, which could only verify two cases for the CIFAR10 small Sigmoid network, δ = 0.014. However, it is the fastest method that can verify neural networks up to 3528 times faster than the star set and up to 46 times faster than DeepPoly in our evaluation.