Laplacian networks: bounding indicator function smoothness for neural networks robustness
Laplacian networks: bounding indicator function smoothness for neural networks robustness
复制标题
DOI：
10.1017/atsip.2021.2
复制
发表时间：
2018-05
影响因子：
3.2
通讯作者：
C. Lassance;Vincent Gripon;Antonio Ortega
中科院分区：
文献类型：
--
作者：
C. Lassance;Vincent Gripon;Antonio Ortega
在过去的几年里，深度学习（DL）的鲁棒性（即当输入受到扰动时保持相同决策的能力）已经成为一个至关重要的问题，特别是在错误分类可能产生严重后果的环境中。为了解决这个问题，作者提出了不同的方法，例如添加正则化器或使用噪声示例进行训练。在本文中，我们介绍了一个正则化的基础上的拉普拉斯算子的相似性图从表示的训练数据在DL架构的每一层。这个正则化器惩罚不同类的例子之间的距离的大变化（跨架构中的连续层），因此强制类边界的平滑变化。我们为这种正则化器提供了理论依据，并证明了它在提高经典监督学习视觉数据集对各种扰动的鲁棒性方面的有效性。我们还表明，它可以与现有的方法相结合，以提高整体的鲁棒性。
For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.