Curriculum Design for Machine Learners in Sequential Decision Tasks
Curriculum Design for Machine Learners in Sequential Decision Tasks
复制标题
DOI：
10.1109/tetci.2018.2829980
复制
发表时间：
2017-05
影响因子：
5.3
通讯作者：
Bei Peng;J. MacGlashan;R. Loftin;M. Littman;David L. Roberts;Matthew E. Taylor
中科院分区：
文献类型：
--
作者：
Bei Peng;J. MacGlashan;R. Loftin;M. Littman;David L. Roberts;Matthew E. Taylor
机器学习领域的现有工作表明，算法可以首先在简单的例子上使用神经网络学习，然后再转向更困难的问题。这项工作研究了顺序决策任务的背景下，不同的课程如何影响学习的推箱子一样的域的问题，并提出了用户研究的结果，探讨非专家是否产生有效的课程。我们的研究结果表明，1）评估反馈的方式给予代理，因为它学习个别任务不影响不同课程的相对质量，2）非专家用户可以成功地设计课程，导致更好的整体性能比有代理从头开始学习，和3）非专家用户可以发现并遵循显着的原则时，选择任务的课程。我们还证明了我们的学习算法可以通过将人们在设计课程时使用的原则进行改进。这项工作让我们深入了解了新的机器学习算法和界面的开发，这些算法和界面可以更好地适应机器或人类创建的课程。
Existing work in machine learning has shown that algorithms can benefit from the use of curricula—learning first on simple examples before moving to more difficult problems. This work studies the curriculum-design problem in the context of sequential decision tasks, analyzing how different curricula affect learning in a Sokoban-like domain, and presenting the results of a user study that explores whether nonexperts generate effective curricula. Our results show that 1) the way in which evaluative feedback is given to the agent as it learns individual tasks does not affect the relative quality of different curricula, 2) nonexpert users can successfully design curricula that result in better overall performance than having the agent learn from scratch, and 3) nonexpert users can discover and follow salient principles when selecting tasks in a curriculum. We also demonstrate that our curriculum-learning algorithm can be improved by incorporating the principles people use when designing curricula. This work gives us insights into the development of new machine-learning algorithms and interfaces that can better accommodate machine- or human-created curricula.