On the Statistical Benefits of Curriculum Learning
On the Statistical Benefits of Curriculum Learning
复制标题
DOI：
--
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Ziping Xu;Ambuj Tewari
中科院分区：
文献类型：
--
作者：
Ziping Xu;Ambuj Tewari
课程学习（CL）是一种常用的机器学习训练策略。然而，我们对 CL 的好处仍然缺乏清晰的理论理解。在本文中，我们研究了 CL 在结构化和非结构化设置下的多任务线性回归问题中的优势。对于这两种设置，我们得出了具有提供最佳课程的预言机和没有预言机（代理必须自适应地学习良好课程）的 CL 的极小极大率。我们的结果表明，在非结构化环境中，自适应学习从根本上来说比预言学习要困难，但它只是在结构化环境中引入了一个小的额外术语。为了将理论与实践联系起来，我们为一种流行的经验方法提供了理由，该方法通过将其保证与上述极小极大率进行比较来选择具有最高局部预测增益的任务。
Curriculum learning (CL) is a commonly used machine learning training strategy. However, we still lack a clear theoretical understanding of CL's benefits. In this paper, we study the benefits of CL in the multitask linear regression problem under both structured and unstructured settings. For both settings, we derive the minimax rates for CL with the oracle that provides the optimal curriculum and without the oracle, where the agent has to adaptively learn a good curriculum. Our results reveal that adaptive learning can be fundamentally harder than the oracle learning in the unstructured setting, but it merely introduces a small extra term in the structured setting. To connect theory with practice, we provide justification for a popular empirical method that selects tasks with highest local prediction gain by comparing its guarantees with the minimax rates mentioned above.