Understanding Self-Paced Learning under Concave Conjugacy Theory
Understanding Self-Paced Learning under Concave Conjugacy Theory
复制标题
DOI：
10.4310/cis.2018.v18.n1.a1
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Shiqi Liu;Zilu Ma;Deyu Meng
中科院分区：
文献类型：
--
作者：
Shiqi Liu;Zilu Ma;Deyu Meng
通过模拟人类/动物从易到难的学习方式，称为课程学习（CL）和自定进度学习（SPL）的学习制度最近得到了研究并引起了广泛的兴趣。然而，分析这种学习机制为何有效的内在机制尚未得到全面研究。针对这个问题，本文提出了一种凹共轭理论来研究 CL/SPL 的见解。具体来说，通过使用该理论，我们证明了 SPL 机制和潜在凹目标的等价性，这与统计和机器学习中广泛使用的已知非凸正则化惩罚密切相关。除了之前解释 CL/SPL 见解的理论之外，这种新的理论框架一方面有利于为某些任务设计新的 SPL 模型的两种直接方法，另一方面可以帮助实现自定进度课程学习的潜在目标，这是 CL/SPL 的高级版本，在一定程度上拥有两种学习机制的优点。这进一步促进了对 SPCL 的理论理解，而不仅仅是传统的 CL/SPL。在这一理论下，我们试图实现偏序课程和小组课程这两种课程形式的内在潜在目标，这很容易遵循相应 SPCL 体系的理论理解。
By simulating the easy-to-hard learning manners of humans/animals, the learning regimes called curriculum learning~(CL) and self-paced learning~(SPL) have been recently investigated and invoked broad interests. However, the intrinsic mechanism for analyzing why such learning regimes can work has not been comprehensively investigated. To this issue, this paper proposes a concave conjugacy theory for looking into the insight of CL/SPL. Specifically, by using this theory, we prove the equivalence of the SPL regime and a latent concave objective, which is closely related to the known non-convex regularized penalty widely used in statistics and machine learning. Beyond the previous theory for explaining CL/SPL insights, this new theoretical framework on one hand facilitates two direct approaches for designing new SPL models for certain tasks, and on the other hand can help conduct the latent objective of self-paced curriculum learning, which is the advanced version of both CL/SPL and possess advantages of both learning regimes to a certain extent. This further facilitates a theoretical understanding for SPCL, instead of only CL/SPL as conventional. Under this theory, we attempt to attain intrinsic latent objectives of two curriculum forms, the partial order and group curriculums, which easily follow the theoretical understanding of the corresponding SPCL regimes.