Towards a Unified Theory of State Abstraction for MDPs
Towards a Unified Theory of State Abstraction for MDPs
复制标题
DOI：
--
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
Lihong Li;Thomas J. Walsh;M. Littman
中科院分区：
文献类型：
--
作者：
Lihong Li;Thomas J. Walsh;M. Littman
状态抽象(或状态聚合)在人工智能和运筹学领域得到了广泛的研究。决策者不是在地面状态空间中工作，而是通过忽略无关的状态信息，将状态组作为一个单元来处理，从而在抽象状态空间中更快地找到解。在强化学习和规划的文献中，已经提出和研究了一些抽象的概念，并得出了积极和消极的结果。给出了马尔可夫决策过程状态抽象的统一处理方法。我们研究了五种特定的抽象方案，其中一些已经被以不同的形式提出，并分析了它们对于规划和学习的可用性。
State abstraction (or state aggregation) has been extensively studied in the fields of artificial intelligence and operations research. Instead of working in the ground state space, the decision maker usually finds solutions in the abstract state space much faster by treating groups of states as a unit by ignoring irrelevant state information. A number of abstractions have been proposed and studied in the reinforcement-learning and planning literatures, and positive and negative results are known. We provide a unified treatment of state abstraction for Markov decision processes. We study five particular abstraction schemes, some of which have been proposed in the past in different forms, and analyze their usability for planning and learning.