Scaling Reinforcement Learning by Adaptive Task Selection and Linear Solution Merging

所提出的研究的目的是研究自治代理如何能够适应动态部分已知的任务环境。 这种代理的潜在应用范围从自动交付家务的硬件机器人到从互联网检索信息的软件程序。 本研究将集中在自适应控制 这就是所谓的强化学习 在这种方法中，智能体通过试验和错误来获得任务技能 通过 选择 行动 的 最大 一 奖励函数。强化学习有一些问题。 它的收敛速度非常慢，特别是在奖励很少出现的大状态空间问题中。此外，学习到的技能在相关任务之间的转移也很差。 本研究将研究使用一种新的模块化任务架构来克服强化学习的这些局限性。该架构将复合多目标任务分解为实现每个目标的原始子任务。 它利用训练时间更多 通过基于任务的难度和重要性在学习不同任务之间动态切换来有效地学习。 它通过使用加权线性和函数将学习到的解决方案重用到原始子任务来增加任务之间的传输。 它通过使用强化学习方法来更有效地解决重复性任务， 奖励 一个详细的实验研究的建议 架构 将 进行，使用各种模拟和真实的机器人试验台。

The aim of the proposed research is to study how autonomous agents can adapt to dynamic partially known task environments. Potential applications of such agents range from hardware robots that automate delivery chores to software programs that retrieve information from the Internet. This research will focus on an adaptive control paradigm called reinforcement learning. In this approach, agents acquire task skills through trial and error by selecting actions that maximize a reward function.Reinforcement learning has some problems. It converges extremely slowly, especially in large state space problems where rewards occur infrequently. Also, the learned skills transfer poorly across related tasks. This research will investigate using a novel modular task architecture to overcome these limitations of reinforcement learning. The proposed architecture decomposes composite multiple goal tasks into primitive subtasks that achieve each individual goal. It utilizes training time more efficiently by dynamically switching between learning different tasks based on their difficulty and importance. It increases transfer across tasks by reusing solutions learned to primitive subtasks using a weighted linear sum function. It solves recurrent tasks more effectively by using a reinforcement learning method that optimizes average reward. A detailed experimental study of the proposed architecture will be undertaken, using a variety of simulated and real robot testbeds.