O N THE C ERTIFIED R OBUSTNESS FOR E NSEMBLE M ODELS AND B EYOND-猫眼课题宝

DOI：

发表时间：

2022

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Zhuolin Yang;Linyi Li;Xiaojun Xu;B. Kailkhura;Tao Xie;Bo Li

表明深度神经网络(DNN)很容易受到敌意的攻击，目的是通过添加小幅度的扰动来误导DNN。为了防御这样的攻击，针对单个ML模型，已经广泛地研究了经验和理论防御方法。在这项工作中，我们旨在分析和提供集成ML模型的确定性稳健性，以及对于不同的集成协议的稳健性的充要条件。尽管从经验上看，集成模型比单个模型更稳健，但令人惊讶的是，我们ﬁ发现，在确定的稳健性方面，标准集成模型与单个模型相比只取得了轻微的改善。因此，为了探索保证提供确定的ﬁ稳健集成ML模型的条件，我们首先证明了在模型光滑性的假设下，偏离梯度和大的残差是保证确定的ﬁ稳健集成模型的充要条件。然后，基于所提出的先集成后平滑策略，给出了有界模型光滑性分析。我们还证明了在温和的条件下，集成模型总是可以获得比单一基础模型更高的确定稳健性。受ﬁ理论的启发，我们提出了轻量级分集正则化训练方法来训练ﬁ稳健的集成ML模型。大量的实验表明，与现有的单一和集成ML模型相比，我们的动态随机响应增强型集成能够始终如一地实现更高的确定性稳健性，证明了最先进的Certiﬁed L 2在MNIST，CIFAR-10模型上的稳健性。对于我们系综中的基础模型，我们遵循基线中使用的Conﬁ公式：分别针对MNIST、CIFAR-10和ImageNet数据集的LeNet(Le-Cun等人，1998年)、Resnet-110和Resnet-50(他等人，2016)。在整个实验过程中，我们使用N=3个基本模型来构建用于演示的系综。我们预计，更多的基础模型将产生更高的整体稳健性。训练出一支健壮的合唱团。我们的分析为基于正规化的训练方法提供了公正的ﬁ基础。大量的实验表明，与现有的基准相比，DRT增强的集成获得了最高的确定性ﬁ稳健性。

show deep neural networks (DNN) are vulnerable to adversarial which aim to mislead DNNs by adding perturbations with small magnitude. To defend against such attacks, both empirical and theoretical defense approaches have been extensively studied for a single ML model . In this work, we aim to analyze and provide the certiﬁed robustness for ensemble ML models , together with the sufﬁcient and necessary conditions of robustness for different ensemble protocols. Although ensemble models are shown more robust than a single model empirically; surprisingly, we ﬁnd that in terms of the certiﬁed robustness the standard ensemble models only achieve marginal improvement compared to a single model. Thus, to explore the conditions that guarantee to provide certiﬁably robust ensemble ML models, we ﬁrst prove that diversiﬁed gradient and large conﬁdence margin are sufﬁcient and necessary conditions for certiﬁably robust ensemble models under the model-smoothness assumption. We then provide the bounded model-smoothness analysis based on the proposed Ensemble-before-Smoothing strategy. We also prove that an ensemble model can always achieve higher certiﬁed robustness than a single base model under mild conditions. Inspired by the theoretical ﬁndings, we propose the lightweight Diversity Regularized Training (DRT) to train certiﬁably robust ensemble ML models. Extensive experiments show that our DRT enhanced ensembles can consistently achieve higher certiﬁed robustness than existing single and ensemble ML models, demonstrating the state-of-the-art certiﬁed L 2 -robustness on MNIST, CIFAR-10, Models. For base models in our ensemble, we follow the conﬁgurations used in baselines: LeNet (Le-Cun et al., 1998), ResNet-110, and ResNet-50 (He et al., 2016) for MNIST, CIFAR-10, and ImageNet datasets respectively. Throughout the experiments, we use N = 3 base models to construct the ensemble for demonstration. We expect more base models would yield higher ensemble robustness. for training a robust ensemble. Our analysis provided the justiﬁcation of the regularization-based training approach DRT. Extensive experiments showed that DRT-enhanced ensembles achieve the highest certiﬁed robustness compared with existing baselines.

O N THE C ERTIFIED R OBUSTNESS FOR E NSEMBLE M ODELS AND B EYOND

文献摘要

相似文献