Proximal Mapping for Deep Regularization
Proximal Mapping for Deep Regularization
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Mao Li;Yingyi Ma;Xinhua Zhang
中科院分区：
文献类型：
--
作者：
Mao Li;Yingyi Ma;Xinhua Zhang
深度学习成功的基础是有效的正则化，允许对数据中的各种先验进行建模。例如，对抗性扰动的鲁棒性，以及多种模态之间的相关性。然而，大多数正则化器都是根据隐藏层输出来指定的，这些输出本身并不是优化变量。与通过模型权重间接优化它们的流行方法相比，我们建议将邻近映射作为一个新的层插入到深度网络中，直接显式地产生良好的正则化隐藏层输出。由此产生的技术显示连接到内核翘曲和辍学，和新的算法开发了强大的时间学习和多视图建模，都优于国家的最先进的方法。
Underpinning the success of deep learning is effective regularizations that allow a variety of priors in data to be modeled. For example, robustness to adversarial perturbations, and correlations between multiple modalities. However, most regularizers are specified in terms of hidden layer outputs, which are not themselves optimization variables. In contrast to prevalent methods that optimize them indirectly through model weights, we propose inserting proximal mapping as a new layer to the deep network, which directly and explicitly produces well regularized hidden layer outputs. The resulting technique is shown well connected to kernel warping and dropout, and novel algorithms were developed for robust temporal learning and multiview modeling, both outperforming state-of-the-art methods.