State Abstractions for Lifelong Reinforcement Learning
State Abstractions for Lifelong Reinforcement Learning
复制标题
DOI：
--
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
David Abel;Dilip Arumugam;Lucas Lehnert;M. Littman
中科院分区：
文献类型：
--
作者：
David Abel;Dilip Arumugam;Lucas Lehnert;M. Littman
在终生的强化学习中，代理人必须在任务中有效地转移知识，同时解决探索，信用分配和概括，可以通过压缩代理商使用的代表来克服这些障碍，从而减少计算机和统计的学习在这一目的中，我们开发了在终生增强学习中计算和使用状态抽象的理论。摘要：（1）及时的摘要，可以计算其最佳形式，以及（2）PAC状态摘要，它们可以保证与任务的分布相关。可以有效地获取，保持在最佳行为附近，并实验降低简单域中的样品复杂性，从而产生一个理想的抽象来用于终身增强学习。与这些积极的结果一起，我们表明在某些情况下，州抽象会对性能产生负面影响。
In lifelong reinforcement learning, agents must effectively transfer knowledge across tasks while simultaneously addressing exploration, credit assignment, and generalization. State abstraction can help overcome these hurdles by compressing the representation used by an agent, thereby reducing the computational and statistical burdens of learning. To this end, we here develop theory to compute and use state abstractions in lifelong reinforcement learning. We introduce two new classes of abstractions: (1) transitive state abstractions, whose optimal form can be computed effi-ciently, and (2) PAC state abstractions, which are guaranteed to hold with respect to a distribution of tasks. We show that the joint family of transitive PAC abstractions can be acquired efficiently, preserve near optimal-behavior, and experimentally reduce sample complexity in simple domains, thereby yielding a family of desirable abstractions for use in lifelong reinforcement learning. Along with these positive results, we show that there are pathological cases where state abstractions can negatively impact performance.