A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM
A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM
复制标题
DOI：
10.1109/smc.2018.00284
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Peifeng Liang;X. Yao;Jinglu Hu
中科院分区：
文献类型：
--
作者：
Peifeng Liang;X. Yao;Jinglu Hu
门控线性网络能够模拟具有复合激活函数R（x）= x * S（x）的预训练神经网络的功能。然后可以用公式表示SVM以进一步隐式地优化门控线性网络，其中通过使用从预训练的神经网络生成的门信号S（x）来组成准线性核。通过这种方式，我们实现了基于核学习的神经网络。本文将距离度量学习应用于核学习的改进。在神经网络的预训练中，采用距离度量学习的损失函数作为正则化项。通过引入距离度量学习的损失函数，使类内样本更近，类间样本更远，从而提高了拟线性核的性能。因此，采用准线性核的SVM优化的分类器将具有更好的性能。将所提出的分类方法应用于不同的现实数据集，仿真结果证实了所提出方法的有效性。
A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x * S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.