A theoretical understanding of self-paced learning
A theoretical understanding of self-paced learning
复制标题
DOI：
10.1016/j.ins.2017.05.043
复制
发表时间：
2017-11
期刊：
影响因子：
--
通讯作者：
Deyu Meng;Qian Zhao;Lu Jiang
中科院分区：
文献类型：
--
作者：
Deyu Meng;Qian Zhao;Lu Jiang
自主学习（SPL）是最近提出的一种通过模仿人类/动物的学习原理而设计的方法。各种SPL的实现方案已被设计用于不同的计算机视觉和模式识别任务，并在这些应用中的经验证明是有效的。然而，文献中对SPL缺乏理论上的认识。针对这一研究空白，本研究试图对SPL方案提供一些新的理论认识。具体来说，我们证明了SPL上的解决方案策略符合优化最小化算法上实现的隐式目标函数。此外，我们发现这个隐式目标中包含的损失函数与统计学和机器学习中已知的非凸正则化惩罚（NCRP）具有类似的配置。这种联系启发我们发现SPL机制和NCRP形式之间更多的内在联系，如平滑剪切绝对偏差（SCAD），对数惩罚（EXP）和非凸指数惩罚（EXP）。然后可以很好地解释SPL下的鲁棒性的见解。我们还分析了SPL的能力，就其容易损失的事先嵌入属性，并提供了一个有见地的解释，在当前SPL的变化的有效性机制。此外，我们还设计了一个群偏序损失先验，这对于弱标记的大规模数据处理任务特别有用。通过在FCVID数据集（目前最大的手动注释视频数据集之一）之前应用SPL，我们的方法实现了现有方法的最先进性能，这进一步支持了所提出的理论论点。
Self-paced learning (SPL) is a recently proposed methodology designed by mimicking through the learning principle of humans/animals. A variety of SPL realization schemes have been designed for different computer vision and pattern recognition tasks, and empirically demonstrated to be effective in these applications. However, the literature is in lack of the theoretical understanding of SPL. Regarding this research gap, this study attempts to provide some new theoretical understanding of the SPL scheme. Specifically, we prove that the solution strategy on SPL accords with a majorization minimization algorithm implemented on an implicit objective function. Furthermore, we found that the loss function contained in this implicit objective has a similar configuration with the non-convex regularized penalty (NCRP) known in statistics and machine learning. Such connection inspires us to discover more intrinsic relationships between the SPL regimes and the NCRP forms, like smoothly clipped absolute deviation (SCAD), logarithmic penalty (LOG) and non-convex exponential penalty (EXP). The insight of the robustness under SPL can then be finely explained. We also analyze the capability of SPL regarding its easy loss-prior-embedding property, and provide an insightful interpretation of the effectiveness mechanism under current SPL variations. Moreover, we design a group-partial-order loss prior, which is especially useful for weakly labeled large-scale data processing tasks. By applying SPL with this loss prior to the FCVID dataset, which is currently one of the largest manually annotated video dataset, our method achieves state-of-the-art performance above existing methods, which further supports the proposed theoretical arguments.