Near Optimal Behavior via Approximate State Abstraction
Near Optimal Behavior via Approximate State Abstraction
复制标题
DOI：
--
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
David Abel;D. E. Hershkowitz;M. Littman
中科院分区：
文献类型：
--
作者：
David Abel;D. E. Hershkowitz;M. Littman
可以使用状态抽象来调节组合爆炸的组合爆炸（RL）算法。可以凝结大型任务表示形式，以便保留基本信息，因此，解决方案是可以计算的。但是，仅将完全相同的情况视为同等的精确抽象，在没有两种情况完全相同的环境中没有提供抽象的机会。在这项工作中，我们研究了近似状态抽象，这些概述将几乎相同的情况视为同等的情况。我们提供了从四种类型的近似抽象中得出的行为质量的理论保证。此外，我们从经验上证明，近似抽象会导致任务复杂性的降低和各种环境中行为最优性的有限损失。
The combinatorial explosion that plagues planning and reinforcement learning (RL) algorithms can be moderated using state abstraction. Prohibitively large task representations can be condensed such that essential information is preserved, and consequently, solutions are tractably computable. However, exact abstractions, which treat only fully-identical situations as equivalent, fail to present opportunities for abstraction in environments where no two situations are exactly alike. In this work, we investigate approximate state abstractions, which treat nearly-identical situations as equivalent. We present theoretical guarantees of the quality of behaviors derived from four types of approximate abstractions. Additionally, we empirically demonstrate that approximate abstractions lead to reduction in task complexity and bounded loss of optimality of behavior in a variety of environments.