Robust Representations in Deep Learning
Robust Representations in Deep Learning
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Shu Liu;Qiang Wu
中科院分区：
文献类型：
--
作者：
Shu Liu;Qiang Wu
深度神经网络在机器学习和人工智能中发挥着越来越重要的作用，以处理复杂的数据。深度神经网络的性能在很大程度上取决于网络架构和损失函数。虽然损失函数最常见的选择是回归分析的平方损失，但众所周知，它对离群值和对抗性样本很敏感。为了提高鲁棒性，我们将相关熵损失引入深度神经网络的实现。我们进一步将神经网络架构分为特征提取组件和函数评估组件，并设计了四个两阶段算法来研究哪个组件更受鲁棒损失的影响。在几个真实的数据集上的应用表明，鲁棒深度神经网络可以有效地生成复杂数据的鲁棒表示，两阶段算法始终比一阶段算法更强大。
—Deep neural networks are playing increasing roles in machine learning and artificial intelligence to handle complicated data. The performance of deep neural networks depends highly on the network architecture and the loss function. While the most common choice for loss function is the squared loss for regression analysis it is known to be sensitive to outliers and adversarial samples. To improve the robustness, we introduce the use of the correntropy loss to the implementation of deep neural networks. We further split the neural network architecture into a feature extraction component and function evaluation component and design four two-stage algorithms to study which component is more impacted by the use of the robust loss. The applications in several real data sets indicates that the robust deep neural networks can efficiently generate robust representations of complicated data and the two-stage algorithms are consistently more powerful than their one-stage counterparts.