Synergy-of-Experts: Collaborate to Improve Adversarial Robustness
Synergy-of-Experts: Collaborate to Improve Adversarial Robustness
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Sen Cui;Jingfeng Zhang;Jian Liang;Bo Han;Masashi Sugiyama;Changshui Zhang
中科院分区：
文献类型：
--
作者：
Sen Cui;Jingfeng Zhang;Jian Liang;Bo Han;Masashi Sugiyama;Changshui Zhang
学习对抗鲁棒模型需要对其自然输入的小邻域进行不变预测，经常遇到模型容量不足的情况。有研究表明，在一个集成中学习多个子模型可以缓解这一不足，进一步提高泛化和鲁棒性。然而，该团队基于投票的策略排除了少数人做出正确预测的可能性。因此，本文通过一种协作方案——专家协同（SoE）来进一步改进集成。与基于投票的策略相比，SoE即使只存在一个正确的子模型，也能够实现正确的预测。在国有企业中，每个子模型都适合其特定的漏洞区域，并保留其余子模型以适应其他漏洞区域，从而有效地优化了模型容量的利用率。经验实验证明，SoE在对抗白盒攻击和基于转移的对抗性攻击方面优于各种集成方法。源代码可从https://github.com/cuis15/synergy-of-experts获得。
Learning adversarially robust models requires invariant predictions to a small neighborhood of its natural inputs, often encountering insufficient model capacity . There is research showing that learning multiple sub-models in an ensemble could mitigate this insufficiency, further improving the generalization and the robustness. However, the ensemble’s voting-based strategy excludes the possibility that the true predictions remain with the minority . Therefore, this paper further improves the ensemble through a collaboration scheme—Synergy-of-Experts (SoE). Compared with the voting-based strategy, the SoE enables the possibility of correct predictions even if there exists a single correct sub-model. In SoE, every sub-model fits its specific vulnerability area and reserves the rest of the sub-models to fit other vulnerability areas, which effectively optimizes the utilization of the model capacity. Empirical experiments verify that SoE outperforms various ensemble methods against white-box and transfer-based adversarial attacks. The source codes are available at https://github.com/cuis15/synergy-of-experts .