Active inference and agency: optimal control without cost functions
Active inference and agency: optimal control without cost functions
复制标题
DOI：
10.1007/s00422-012-0512-8
复制
发表时间：
2012-10-01
影响因子：
1.9
通讯作者：
Montague, Read
中科院分区：
文献类型：
--
作者：
Friston, Karl J.;Samothrakis, Spyridon;Montague, Read
本文描述了不确定条件下（部分可观测）马尔可夫决策问题的变分自由能公式。我们证明了最优控制可以转换为主动推理。在主动推理中，在生成模型下，关于隐藏状态的行动和后验信念都最小化了观察状态负对数似然的自由能界。在这种情况下，奖励或成本函数被吸收到关于状态转换和最终状态的先验信念中。有效地，这将最优控制转化为一个纯粹的推理问题，使标准贝叶斯滤波技术的应用成为可能。然后，我们考虑基于对未来隐藏状态的后验信念的最优轨迹。至关重要的是，这需要将控制建模为一种隐藏状态，使生成模型具有代理的表示。这导致了对隐藏控制状态进行推理和不进行推理的模型之间的区别；分别是无代理模型和基于代理模型。
This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.