Discovering Relational Domain Features for Probabilistic Planning
Discovering Relational Domain Features for Probabilistic Planning
复制标题
发现概率规划的关系域特征
DOI：
--
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
R. Givan
中科院分区：
文献类型：
--
作者：
Jia;R. Givan
在马尔可夫决策过程的序贯决策问题中，使用域特征的状态值函数近似是扩大可行问题规模的关键技术。我们考虑的问题，自动发现有用的域功能的问题域表现出关系结构。具体来说，我们考虑学习紧凑的关系特征，而不需要人类专业知识的输入;我们既不使用专家决策，也不使用人类领域知识，而不仅仅是基本的领域定义。我们提出了一种学习线性值函数表示的关系特征的方法-数值特征通过其与当前值函数的Bellman残差的拟合来选择，并在需要时自动学习并添加到表示中。从值函数表示中的一个平凡特征开始，我们的方法通过将特征学习与近似值迭代相结合来找到有用的值函数。俄罗斯方块和概率规划竞争领域的实证研究表明，我们的技术代表了最先进的领域独立的特征学习和随机规划在关系域。
In sequential decision-making problems formulated as Markov decision processes, state-value function approximation using domain features is a critical technique for scaling up the feasible problem size. We consider the problem of automatically finding useful domain features in problem domains that exhibit relational structure. Specifically we consider learning compact relational features without input from human expertise; we use neither expert decisions nor human domain knowledge beyond the basic domain definition. We propose a method to learn relational features for a linear value-function representation—numerically valued features are selected by their fit to the Bellman residual of the current value function and are automatically learned and added to the representation when needed. Starting with only a trivial feature in the value-function representation, our method finds useful value functions by combining feature learning with approximate value iteration. Empirical work presented here for Tetris and for probabilistic planning competition domains shows that our technique represents the state-of-the-art for both domain-independent feature learning and for stochastic planning in relational domains.